DOCUMENT RESUME 



ED 385 704 



CE 069 583 



AUTHOR 
TITLE 

INSTITUTION 
SPONS AGENCY 



PUB DATE 
NOTE 

PUB TYPE 

EDRS PRICE 
DESCRIPTORS 



IDENTIFIERS 



Kemple, James J»; And Others 

Florida* s Project Independence, Benefits, Costs, and 
Two-Year Impacts of Florida* s JOBS Program, 
Manpower Demonstration Research Corp,, New York, 
N, Y, 

Department of Health and Human Services, Washington, 
D,C,; Florida State Dept, of Health and 
Rehabilitative Services, Tallahassee,; Ford 
Foundation, New York, N,Y, 
Apr 95 

287p,; For the first-year report, see ED 366 804, 
Reports ~ Evaluative/Feasibility (142) 

MF01/PC12 Plus Postage, 

Adult Education; Cost Effectiveness ; '^Employment 
Services; Federal Legislation; '^Job Placement; '''Job 
Training; '^'One Parent Family; Program Effectiveness; 
Program Evaluation; State Programs; Wages; '"'Welfare 
Recipients 

Florida; Job Opportunities and Basic Skills 
Program 



ABSTRACT 

An evaluation addressed whether Project Independence, 
Florida's Job Opportunities and Basic Skills Program, had reduced 
welfare costs and increased the employment, earnings, and income of 
welfare recipients. It examined the experiences of more than 18,000 
Aid to Families with Dependent Children (AFDC) applicants and 
recipients from 9 randomly chosen counties in Florida between July 
1990 and August 1991, Each sample member was assigned at random to 
either a program or control group. Information about their 
employment, earnings, AFDC receipt, and Food Stamp receipt was 
collected for 2-3 years. Findings revealed both the program's 
potential and limitations. Over a 2-year follow-up period. Project 
Independence produced average AFDC savings of $265 per program group 
member. It increased average 2-year earnings by only $227 per program 
group member. During the evaluation period, the project evolved from 
its original design to a program with fewer resources to spend on the 
typical participant. For sample members who were referred to the 
program early on during its intended version, the program produced 
2-year AFDC savings averaging $241 and 2-year earnings gains 
averaging $439 per program group member. For those referred to the 
program later, the program produced 2-year AFDC savings similar to 
those for the early group, but almost no earnings gain, (Appendixes 
include supplemental tables, response analysis, information on 
estimating participation rates, and a list of 24 references,) 
(YLB) 



Vc Vc 5'c Vc i( Vc Vc Vc Vc Vc ^'c Vc Vc Vc Vc Vc Vc Vc Vc Vc Vc Vc Vc Vc Vc Vc Vc Vc Vc i< Vc Vc Vc Vc Vc Vc ^'c Vc Vc Vc Vc Vc Vc Vc iV I'c i: Vc Vc Vc Vc Vc Vc Vc Vc Vc Vc •>'< Vc Vc •)'< Vc Vc Vc it V? 

Reproductions supplied by EDRS are the best that can be made 
from the original document, 

Vc Vc ?'c Vc Vc Vc Vc Vc Vc Vc Vc V; Vc Vc Vc Vc Vc Vc Vc Vc Vc Vc Vc Vc Vc V? Vc Vc Vc Vc Vc Vc Vc Vc Vc Vc Vc Vc Vc Vc Vc Vc it Vc Vc Vc Vc Vc Vc Vc Vf Vc Vc Vc Vc Vc Vc Vc Vc Vc Vc Vc Vc Vc Vc Vc Vc 



Florida's Project Independence 



Benefits, Costs, and 
Two-Year Impacts of 
Florida's JOBS Program 



James J. Kerr^le 
Daniel Frigmande] 
^VfeuDnica^^ellera^^ 



April 1995^^ - 

Manpower Demonstration 
/'dPlesearch Corporation 



BOARD OF DIRECTORS 

RICHARD P. NATHAN, Chairman 
Provost, Rockefeller College 
State University' of New York 
Director, Rockefeller Institute 
of Government 

PAUL H. O'NEILL. Immmr 
Chairman and CEO 
;\lcoa 

ELI GINZBERG, Chairman Emeritus 
Director 

The Eisenhower Center tor the 

(x)nseiTation of Human Resources 
CoUimbia University 



REBECCA M. BLANK 

Professor of Economics 
N o r thwes te r n U n i\e rs i ty 

ANTONIA HERNANDEZ 

President and General Counsel 
Mexican American Legal Defense and 
Educational Fund 

ALAN KISTLER 

President 

Human Resources Development Institute 
AFL-C:iO 

RICHARD J. MURNANE 

Professor oi Education 
Graduate School of Education 
Hanard Uuiveisitv 



RUDOLPH G. PENNER 

Director of Economic Studies 
Policy Economics Group 
KPMG Peat Mamick 

FRANKUN D. RAINES 

Vice Chairman 
Fannie Mae 

ROBERT SOLOW 

Institute Professor 

Massachusetts Institute of Technology- 

GIIJ5ERT STEINER 

Senior Fellow 
Brookings Institution 

MITCHELL SVIRIDOFT 

Professor Emeritus and Senior Fellow 
Community- Development Research Ck'nter 
New School for Social Research 

WILLIAM JULIUS WILSON 

Lucy Flower University Professor 
of Sociology and Public Policv 
The University of Chicago 

WILUAM S. WOODSIDE 

Chairman, Sky Chefs, Inc. 
Former Chairman and C^EO, 
Primerica Ck>rporation 



JUDITH M, GUERON 
President 

Manpower Demonstration Research ( Corporation 



MDRC 

3 



Florida's Project Independence 

Benefits, Costs, and 
Two-Year Impacts of 
riorida^s JOBS Program 



James J* Kemple 
Daniel Friedlander 
Veronica Felldrath 



April 1995 

Manpower Demonstration 
Research Corporation 



MDRC 



ERIC 



4 



The Manpower Demonstration Research Corporation's evaluation of Florida's Project 
Independence Program is funded by a contract with Florida's State Department of Health 
and Rehabilitative Services and with support from the Ford Foundation and the U.S. 
Department of Health and Human Services. 



Dissemination of MDRC publications is also supported by our Public Policy Outreach 
funders: 



Ford Foundation 

The Ambrose Monell Foundation 
Alcoa Foundation 
Aetna Foundation, Inc. 
Bristol-Myers Squibb Foundation, Inc. 



Metropolitan Life Foundation 
The Prudential Foundation 
The Union Carbide Foundation, Inc. 
The Travelers Foundation 



The findings and conclusions in this report do not necessarily represent the official positions or 
policies of the funders. 



Library of Congress Cataloging-in-Publication Data 
Kemple, James J. 

Florida's Project Independence : benefits, costs, and two-year impacts of 
Florida's JOBS program / James J. Kemple, Daniel Friedlander, Veronica 
Fellerath. 

p. cm. 

"April 1995." 

Includes bibliographical references. 

1. Welfare recipients — Employment — Florida. 2. Welfare recipients — 
Training of — Florida. 3. Welfare recipients — Education — Florida. I. Friedlander, 
Daniel. II. Fellerath, Veronica. III. Title. 
HV98.F5K465 1995 

362.5'8'09759— dc20 95-2800 

CIP 



Copyright C 1995 by the Manpower Demonstration Research Corporation 



ERIC 



5 



ACKNOWLEDGMENTS 



A study of the scope and magnitude of the Project Independence evaluation is a true partnership 
and would not have been possible without the unstinting conmiitment of many people at all levels of 
Florida's Department of Health and Rehabilitative Services (HRS) and staff at Project Independence. 

Implementation of the research design required the efforts of hundreds of Public Assistance 
Specialists, whose capable handling of the random assignment procedures - in addition to their 
numerous other responsibilities - built the foundation for the whole study. They were supported by 
their unit supervisors and district Economic Services Program Administrators. The district Project 
Independence Program Administrators aided die evaluation in many ways and ensured that the MDRC 
research team had access to their staff in order to obtain information about the program. These 
individuals include: Barbara Coles, Isabell Davis, Greg Frohnappel, Bob Janssen, Judy Jones, Terry 
Keeter, Dan Miller, Herb Moore, and Martha Pepper. The study also benefited from the assistance 
of the many (too numerous to mention by name) Project Independence unit supervisors, employment 
counselors, employment specialists, and support staff who permitted us to interview them, observe 
their programs, and collect information important to this report. 

From HRS headquarters in Tallahassee, Albertine McDaniel provided strong support for the 
evaluation and offered useful comments on an earlier draft of this report. We are grateful to Don 
Winstead, who assured the evaluation's success during a critical period. We owe a special debt to Dan 
Gcss, who worked with the counties to implement the research design and reviewed this report, and 
to Bill Hudgens, who was instrumental in helping MDRC gain access to the various data sources used. 
V, Sheffield Kenyon, Thomas Arnold, Calvin Melton, James Clark, and Reginald Smith made 
important contributions to the study's success. Tracy Sumner (now with Florida's Department of 
Labor and Employment Security, LES) and Judy Moon provided helpful insights into the operational 
details of Project Independence and commented on drafts of this report. Several individuals made 
valuable contributions to the cost study, including Sue Zwirz from the HRS Office of Revenue 
Management, Fred Dietrich from the Project Independence Bureau at LES, Sandra Harris and Desiree 
Reddick from Florida's Office of Children and Family Services, and Mark Eggers and Lee Young 
from the State's Department of Education. Members of the Project Independence Evaluation Advisory 
Committee - Ken Baer, Bob Barrios, Budd Bell, Cyndi Craig, Rowe Hinton, Cindy Huddleston, Bill 
Nugent, Phil Reeves, Margarita Romo, Deborah Sims, Sandy Steen, and Marcia Weider - offered 
thoughtful insights and suggestions. We would also like to acknowledge the suppon of HRS 
Secretaries H. James Towey, Robert Williams, and Gregory Coler. 

Good data provide the infrastructure for evaluations. We are grateful to Billy Davis and Jerry 
Arnold at Florida's Office of the Auditor General, Division of Public Assistance Fraud (DPAF), whose 
professionalism and perseverance facilitated MDRC's access to automated AFDC, Food Stamp, and 
Unemployment Insurance records. Joe Franklin at HRS provided helpful information about the 
FLORIDA System. 

Many of our colleagues at MDRC made important contributions to both the substance and 
production of this report. Judith Gueron, Gordon Berlin, and Barbara Goldman provided overall 
guidance for the entire evaluation effort and were senior reviewers of the reports, James Riccio was 
a senior reviewer of this report and also made substantive contributions to the cost and benefit-cost 

-iii- 



ERIC 



6 



analyses. Christopher Bost assisted with the impact and benefit-cost analyses. Jean Knab provided 
research assistance on all aspects of this report and coordinated its production. Stacey Fox and Linda 
Choi assisted with the cost analysis. Electra Small and Vince Hayden coordinated the data processing. 
Marilyn Price was MDRC's primary liaison with HRS administration and staff in Florida. 

Karen Paget managed the collection and processing of the data and provided valuable guidance. 
Joel Gordon also played a key role in this effort. Adria Gallup-Black was MPRC's data liaison with 
HRS and DPAF and developed all the data files for this report. Mohammad Amzad programmed the 
administrative data. Anita Kraus and Gaston Murray programmed the survey data. 

We wish to acknowledge the useful comments and insights of members of MDRC*s Welfare 
Studies Committee: Robert Solow (Chairman), Henry Aaron, Rebecca M. Blank, Gary Burtless, 
Linda Datcher-Loury, Mark Greenberg, Frank Levy, Richard J. Mumane, Richard P. Nathan, and 
Audrey Rowe. 

Judith Greissman edited the report. Word processing was done by Patt Pontevolpe and 
Stephanie Cowell. 



The Authors 



ERIC 



-iv- 

7 



PREFACE 



This is the final report on a five-year evaluation of Florida's statewide Project Independence 
program - Florida's version of the current federal-state Job Opportunities and Basic Skills Training 
(JOBS) Program. It adds to the growing body of knowledge on the success of different approaches 
to requiring or assisting welfare recipients to become self-sufficient. The study was conducted by the 
Manpower Demonstration Research Corporation and funded by the Florida Department of Health and 
Rehabilitative Services, the Ford Foundation, and the U.S. Department of Health and Human Services. 

Large-scale, rigorous, longitudinal studies of real-world operating programs like Project 
Independence require the dedication and involvement of many people at the state and local levels, 
people whose primary responsibility is operating public programs, not collaborating as partners on a 
major evaluation. The study was launched because of the state's interest in learning whether it was 
operating this major public program effectively. Over the past five years, many busy administrators 
and their staff made a commitment to opening ^'heir programs to examination, implementing research 
protocols, and providing data. It is our hope and belief that the findings — and their relevance to 
Florida's new welfare reform initiative, the Family Transition Program — suggest the wisdom of 
undertaking such an effort. 

The results point to both the strengths and limitations of the Project Independence approach, 
a relatively low-cost one focused on getting people into jobs quickly. They also speak to the challenge 
of implementing programs in a period when caseloads grow as economic conditions worsen. The study 
(which, for reasons outlined in the report, provides a conservative estimate of the program's 
accomplishment) shows that Project Independence modestly increased welfare recipients' employment 
and earnings and reduced their reliance on welfare. Moreover, it accomplished this at no net cost to 
taxpayers, an unusual achievement for a government program. 

Project Independence was most successful in increasing people's earnings when it was 
implemented as intended; success was sharply reduced (particularly for women with young children) 
when welfare caseloads rose and child care and other resources became stretched. Given the fiscal 
pressures states currently face, and may confront even more starkly in the future, these results provide 
a timely warning about the risks of attenuating a program's services to the point where they fall below 
a threshold level at which they can be effective. The benefit-cost results challenge administrators to 
confront the difficult task of reducing dependency among women with very young children. Project 
Independence was clearly cost-effective for women with school-age children, saving taxpayers (over 
five years) more than $1.50 for every dollar spent on the program. But for women with younger 
children (for whom child care outlays were higher and the program's achievements smaller), taxpayers 
lost money and welfare families had less income. With the welfare debate in Florida and the nation 
focused on reducing long-term dependen^^y, the present results suggest the difficulty of achieving 
lasting effects for mothers of young children. 

At a time when the public is demanding that government use resources efficiently, it is 
impressive that Project Independence more than repaid the public's investment. But the large number 
of people remaining on welfare despite participating in the program suggests the clear challenge to 
states interested in imposing some form of time limit on welfare after which people would have to 
work or might receive no further support. 

Judith M. Gueron 
President 
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EXECUTIVE SUMMARY 



Nearly all national welfare reform initiatives since 1971 have called on states to implement a 
quid pro quo arrangement between welfare recipients and the agencies that provide financial assistance 
for poor families. With varying degrees of enforcement and coverage, welfare recipients have been 
required to take concrete steps toward self-sufficiency while government-funded programs provide 
them with support and opportunities to find jobs and leave the rolls. In 1988, the U.S. Congress 
passed the Family Support Act (FSA), dramatically expanding the vision and scope of the obligations 
of welfare families and tlie government. As the centerpiece of FSA, the Job Opportunities and Basic 
Skills Training (JOBS) Program provided new federal matching funds for state wel fare-to- work 
initiatives aimed at increasing the employment and self-sufficiency of applicants to and recipients of 
Aid to Families with Dependent Children (AFDC), the major federal and state cash welfare program. 
To bolster this additional investment, JOBS extended the mandates to AFDC applicants and recipients 
with children as young as three years old (one year old, at state option) and strengthened the rules 
requiring all participants to take jobs or engage in activities leading to employment. However, it left 
states with considerable flexibility to design service delivery strategies and to set priorities for the level 
and emphasis of their investment in JOBS. 

Project Independence is Florida's JOBS program. In creating Project Independence, the 
Florida Employment Opportunity Act of 1987, and the minor modifications that brought the program 
into compliance with FSA in 1988, adopted a broad-coverage and relatively low-cost approach to 
implementing JOBS. Participation requirements cover all of the state's single parents (the vast 
majority of them women) with children age three or older who apply for or receive AFDC and are 
not otherwise exempt. Other states chose to target particular segments of the JOBS-mandatory AFDC 
population, such as those with minimal levels of education or work experiences or those with long 
histories of prior AFDC receipt. In an effort to stretch available resources over its full mandatory 
caseload, Project Independence emphasized relatively low-cost, independent job search services for 
the majority of its participants and reserved its more expensive education and training services for 
tliose considered least able to find work on their own. 

The Manpower Demonstration Research Corporation (MDRC) has been evaluating Project 
Independence under a contract with Florida's Department of Healtli and Rehabilitative Services (HRS) 
and with support from the Ford Foundation and the U.S. Department of Health and Human Services. 
This is the evaluation's second and final report. The central question it addresses is whether Project 
Independence has reduced welfare costs and increased the employment, earnings, and income of 
welfare recipients. Until now, there has been little evidence about whether the mutual obligations 
established by JOBS, and the implementation approach adopted by Project Independence, will translate 
into welfare savings and increased income for the large group of AFDC applicants and recipients with 
preschool-age children (i.e. , between the ages of three and five). The Project Independence evaluation 
is the first designed specifically to learn about this group, as well as about those whose youngest child 
is age six or older. This report also compares the program's costs with the benefits it produced for 
individuals referred to the program, for the government budgets that fund ±e program, and for society 
as a whole. Finally, the report provides important evidence about how resource commitments and 
changes in local contexts are likely to have affected Project Independence's performance. 
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The evaluation addresses these questions by examining the experiences of more than 18,000 
AFDC applicants and recipients from nine randomly chosen counties in Florida. This research sample 
includes all individuals who were determined to meet the state's criteria for mandatory participation 
in Project Independence between July 1990 and August 1991. Each san^iple member was assigned at 
random to either a program group (who were subject to Project Independence's participation 
requirements) or to a control group (who were told that they were not required or permitted to 
participate in Project Independence but could seek services in the community on their own). 
Information about their employment, earnings, AFDC receipt, and Food Stamp receipt was collected 
for a period of at least two years, and for as much as three years, from the time each person was 
randomly assigned to one of the two research groups. Differences between the two groups in these 
outcomes - referred to as the program^s "effects" or "impacts" - reflect the increases or decreases 
caused by Project Independence. For the reasons discussed below, the reported results are likely to 
somewhat underestimate the magnitude of the impacts. 

An Overview of the Findings 

Results from the Project Independence evaluation were mixed, and the findings reveal both the 
program's potential and its limitations. Over a two-year follow-up period; for the full evaluation 
sample, Project Independence produced average AFDC savings of $265 per program group member. 
(In other words, the average program group member received that much less than the average control 
group member.) This represents a 6 percent reduction relative to the control group average, and in 
percentage terms is in the same range as the savings found for single parents in previous evaluations 
of large-scale welfare-to-work programs. Project Independence was orHy marginally successful in 
producing earnings gains. For the full sample, the program increased average two-year earnings by 
only $227 per program group member, a 4 percent gain relative to the control group average. This 
is a small percentage increase compared to those found in other studies. 

During the evaluation period (July 1990 through September 1993), Project Independence 
evolved from its original design to a program that had fewer resources to spend on the typical 
participant. The intended version of the program is most likely reflected in the experiences of sample 
members who were referred to it early on, between July and December 1990. For this "early group," 
the program produced two-year AFDC savings averaging $241 per program group member and two- 
year earnings gains averaging $439 per program group member. Those referred to the program after 
December 1990 (the "late group") experienced Project Independence when it was trying to serve a 
growing caseload v;ith fixed resources and reductions in the availability of child care for Project 
Independence participants. For this late group, the program produced two-year AFDC savings similar 
to those for the early group, but almost no earnings gains. 

Sustained earnings gains were concentrated in one part of the early group: those who had no 
preschool-age children. Their gains reached a level comparable to those found in previous evaluations 
of relatively low-cost welfare-to-work programs. Results were much weaker for sample members with 
preschool-age children. 

Most social programs are conceived primarily as attempts to benefit their participants and not 
as public investments intended to produce government budget savings. Project Independence was 
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intended to reduce public assistance costs - and it did. In fact, savings were large enough to enable 
the program to pay for itself. In other words, for every net dollar spent on providing services for 
program group members (i.e., expenditures beyond the cost of alternative services they would have 
received on their own), the program returned slightly more than one dollar to government budgets 
from reduced welfare payments and increased tax revenues (because of increased earnings). Although 
taxpayers broke even, program participants did not do as well: The small earnings increases produced 
by Project Independence were offset by the larger decreases in AFDC and other welfare payments, 
producing a net financial loss averaging approximately $370 per program group member over the five- 
year period used for the analysis of the program's benefits and costs. However, there were more 
positive findings for women in the early group who had no preschool-age children. For this group, 
the program returned to government budgets $1.55 per net dollar invested, and program participants 
experienced a net financial gain. 

The findings for Project Independence hold particular importance for JOBS program 
administrators and policymakers. First, they suggest that there is no hard-and-fast linkage between 
accomplishing the goal of containing welfare costs by promoting employment and the goal of 
increasing family income. Project Independence decreased the time some people spent on welfare and 
returned enough in welfare savings and increased tax revenues to cover its own cost. Overall, 
however, the program did not increase long-term earnings enough to make most people better off 
financially. Second, if a JOBS program's investment in individual participants falls below a threshold 
level, resources may be insufficient to permit the program to achieve both goals. By enforcing at least 
a short-term participation requirement for a large share of the mandatory caseload. Project 
Independence was able to produce welfare savings even under the stress of increasing caseloads and 
even for women with young children. At its most effective. Project Independence also produced 
sustained earnings increases that offset reductions in welfare receipt, at least for program participants 
with no preschool-age children. However, as limitations on child care and case management resources 
made it increasingly difficult to engage participants intensively, the program's longer-term effect on 
earnings diminished. The lower threshold of program resources was reached with especially serious 
financial effects for women with preschool-age children. 

The Project Independence Model 

Figure 1 is a simplified depiction of the Project Independence program model as it was 
designed by HRS. At the time they came to the Public Assistance Unit (i.e., welfare office) to apply 
for AFDC or to be "redetermined" as eligible to continue receiving it, individuals were informed that 
it was mandatory for them to participate in Project Independence (unless they met the official 
exemption criteria)^ and were referred to an orientation session. Project Independence case managers 



^Exempt individuals include those who are age 60 or older, working 30 or more hours per week at the 
minimum wage, pregnant and in their second or third trimester of pregnancy, permanently ill or incapacitated, 
or required in the home to care for a physically or mentally impaired household member. Project 
Independence also serves AFDC applicants and recipients who are exempt from the program but who wish 
to volunteer, as well as unemployed parents in two-parent households who are required to participate. This 

(continued...) 
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FIGURE 1 

A SIMPLIFIED DEPICTION OF THE PROJECT INDEPENDENCE PROGRAM MODEL 
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used "sanctioning" procedures - a process for reducing the AFDC grants of those who failed to attend 
program activities without a good reason - as the primary mechanism for enforcing the participation 
requirements, which were ongoing for as long as the person remained on welfare. This process was 
used widely, particularly in an effort to engage mandatory participants who did not meet the initial 
requirement of attending a program orientation and then beginning an employment- related activity. 

Project Independence case managers assigned individuals who attended orientation to one of 
two service tracks based on their educational attainment level and recent employment history. As 
illustrated in Figure 1, the first service track began with a two-week independent job search and was 
targeted for those assessed to be "job-ready" according to Project Independence criteria. During the 
period when the research sample was referred by the Public Assistance Units to Project Independence 
(July 1990 through August 1991), the program took an expansive view of job-readiness as completion 
of at least the tenth grade or employment in at least 12 of the previous 36 months. This meant that 
a large majority of those required to participate in the program were considered to be job-ready. 
These criteria, combined with an emphasis on case managers' meeting specific job placement 
standards, reflected Project Independence's employment-focused approach to self-sufficiency. The 
independent job search component required participants to make contact with at least 12 employers to 
apply for a job. This activity was generally unsupervised, although participants were encouraged to 
use the program's job referral services on their own initiative and were required to document their job 
contacts. Participants were often assigned to two or more spells of independent job search before 
moving on to another activity. Those who did not find employment on their own were to be assigned 
to a group job club — a two- to three-week course on how to look for a job, prepare a resume, fill 
out applications, and present oneself in an interview. Following job club, participants were often 
referred to an extended independent job search, which was usually scheduled for four to six weeks. 

The second service track, also shown in Figure 1 , was targeted for the narrowly defined group 
of participants designated "not job-ready" (i.e. , they had not completed the tenth grade and had worked 
in fewer than 12 of the previous 36 months). It was also reserved for job-ready participants who could 
not find work on their own or with the help of job club. The second service track usually began with 
a formal assessment, in which participants were tested for basic literacy skills, discussed their career 
interests with a case manager, and (jointly with the case manager) developed an Employability Plan. 
They were then usually assigned to basic education, post-secondar>' education, or occupational training 
activities provided by local adult schools, community colleges, or proprietary institutions. Basic 
education includes adult basic education, or ABE (i.e., remedial classes in reading and math); classes 
preparing people to take the GED test; or instruction in English as a Second Language (ESL). 
(Passing the GED - General Educational Development - test qualifies a person for a GED certificate, 
which is intended to reflect a basic knowledge of high school subjects.) Post-secondary education 
activities v/ere usually occupationally oriented community college courses. Training activities consisted 
primarily of classroom-based occupational skills training, but could include on-the-job training, usually 
provided through Job Training Partnership Act (JTPA) agencies. People who were already in an 
education or training activity when they entered Project Independence could be allowed to continue 



U... continued) 

report focuses exclusively on single-parent heads of households who were required to participate during the 
period under study; they constituted a majority of the population the program served. 
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in those '*self-initiated" activities, which would be counted as their Project Independence activities. 
Unless they were working at least 30 hours per week, employed mandatory participants were also 
required to engage in one of these activities. 

Project Independence also provided tuition assistance for training or community college classes 
and support services - such ais child care, transportation, tools, and uniforms - considered necessary 
for participants to engage in program activities or to secure employment. Case managers also granted 
short-term deferrals from participation for those who were temporarily unable to engage in program 
activities because of health or family problems or because child care or other needed support services 
were not available. Those placed in the deferral status remained registered in Project Independence 
and were to be contacted at regular intervals to confirm their reason for not being able to participate 
or to re-engage them in activities. This accounts, in part, for the finding that mandatory AFDC 
applicants and recipients were not always active in Project Independence services, as discussed below. 

The Evolution of Project Independence During the Evaluation Period 

The period covered by this evaluation was a time when resources for Project Independence, 
and for JOBS programs in many states, remained fixed but welfare caseloads and unemployment rates 
were increasing. This combination" of circumstances provides a critical context for interpreting the 
present findings. The evaluation began in July 1990, when the first AFDC applicants t \d recipients 
were enrolled in the research sample, and ended in September 1993, when data collection for this 
report was completed, as illustrated in Figure 2. The symbols at the bottom of the figure refer to 
several key events related to changes in Project Independence and the state during the evaluation 
period, and the graphs represent trends in Florida's AFDC caseload and unemployment rate during 
that same period. 

Figure 2 shows that between July 1990 and August 1991 - when AFDC applicants and 
recipients were enrolled in the research sample - Florida's unemployment rate rose from 6.1 to 8.1 
percent (a 33 percent increase) and the state's AFDC caseload grew from approximately 140,000 to 
175,000 (a 25 percent increase). The bottom of Figure 2 also shows that, beginning in early 1991, 
the dramatic growth of the AFDC caseload coincided with a series of state hiring freezes, which were 
budget-induced and prevented Project Independence from increasing its staffing capacity to meet the 
growing demand for its services. Also beginning in January 1991, budget constraints forced HRS to 
restrict the availability of child care for Project Independence participants, particularly those in longer- 
term services such as education and training. Instead, nearly all child care resources had to be 
diverted to employed AFDC recipients and to those entitled to child care after leaving welfare for work 
(i.e., transitional child care). This is significant because mandatory registrants with young children 
could be excused from the participation requirements if child care services were not available. The 
increase in the state's unemployment rate meant that Project Independence's emphasis on moving a 
high percentage of its participants into the labor market as quickly as possible was confronted with a 
growing shortage of job openings. 

Because of all these changes, this report assesses what amounts to two different versions of 
Project Independence, which can be examined through the experiences of individuals who were 
referred to the program during two periods of the evaluation. One group, referred lO in this report 
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FIGURE 2 
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as the "early" group, entered the study between July and December 1990 (as illustrated by the lightly 
shaded portion of Figure 2). This group was exposed to Project Independence when its operations 
more closely matched the model originally intended by HRS. Although the program was a moderate - 
cost one. Project Independence staff had smaller caseloads than they would later have, child care was 
widely available, and the labor market was relatively robust - conditions favorable to the staffs 
ability to enforce the ongoing participation mandate and to help participants find work by facilitating 
their access to employment-related activities and support services. 

The other group (the "late" group) entered the study between January and August 1991, as 
indicated by the darker shaded portion of Figure 2. They were exposed to Project Independence when 
it was forced to spread a fixed set of resources over a rapidly increasing caseload — conditions that 
restricted the staff's capacity to monitor participation and limited the level of support, particularly child 
care, they could provide to help participants obtain job skills and find work. By comparing the 
experiences of these two groups, this report examines whether differences in the program's 
implementation and environment were associated with differences in its effectiveness. It should be 
noted, however, that the program changes were implemented unevenly over time and across counties. 

Figure 2 also shows three changes in Project Independence that occurred after the research 
sample entered the study, and so were not likely to have influenced the findings significantly. Late 
in 1991, HRS modified the Project Independence job-readiness criteria to enable more participants to 
enroll in education or training, rather than independent job search, as their first activity. Also late in 
1991, some counties began taking measures to reduce Project Independence caseloads. The most 
important of these was giving volunteers first preference for the program.. Most program group 
members had already been referred to Project Independence and had attended a program orientation 
by the time these modifications were implemented. Finally, in July 1992, HRS subcontracted 
responsibility for staffing and op'^rating Project Independence at the district level to the Department 
of Labor and Employment Security (LES). Local Project Independence staff remained in place under 
LES and continued to provide case management, monitor participation in independent job search and 
job club activities, and develop linkages with the agencies that provide most of the program's 
employment, education, and training activities. HRS retained overall administrative responsibility for 
the program and serves as its primarj' policymaking agency. The change in departmental 
responsibilities occurred a year after the research sample entered the study and is not likely to have 
affected the evaluation results. 



The Project Independence Evaluation 

The primaiy question for the Project Independence evaluation was whether the program 
increased employment and earnings and reduced AFDC and Food Stamp receipt for those required to 
paiticipate, compared to what their experiences would have been in the absence of this requirement. 
The evaluation was also charged with determining whether Project Independence made individual 
participants better off financially tlmn they would have been without the program and how the 
program's benefits compared to its costs. To answer these questions as reliably and rigorously as 
possible, techniques of randomization were used both to select the nine research counties and to assign 
individuals within the selected counties to one of the two groups into which the research sample was 
divided. Nine counties were randomly selected from the 25 with the largest AFDC caseloads. The 
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'itiiie counties are: Bay (Panama City), Broward (Fort Lauderdale), Dade (Miami), Duval 
(Jacksonville), Hillsborough (Tampa), Lee (Fort Myers), Orange (Orlando), Pinellas (wSt. Petersburg), 
and Volusia (Daytona Beach). Together, they accounted for 58 percent of the state's AFDC caseload 
at the start of the evaluation and include Florida's eight largest cities as well as some suburban and 
rural areas. (Predominantly rural counties with extremely small AFDC caseloads were excluded from 
the study.) The lottery-like method used to select the counties enhanced the evaluation's capacity to 
produce results that can be generalized to at least 90 percent of the state's AFDC caseload that became 
mandatory for Project Independence during the period under study. 

Randomization was also used for the research sample as the basis for determining the 
program's effectiveness - specifically, its effects on employment, earnings, AFDC receipt (i.e., 
months of receipt), and AFDC payments. From July 1990 through August 1991, 18,237 single-parent 
applicants for and recipients of AFDC in the nine research counties were randomly assigned either to 
a "prc;^ram group," which was eligible to receive Project Independence services and was subject to 
the participation mandate, or to a "control group," which did not have access to the program and was 
not subject to the mandate. This type of random assignment research design is widely regarded as the 
most reliable method available for determining the effectiveness of programs such as Project 
Independence. Control group members were given a list of alternative employment and training 
services in the community. They remained eligible for subsidized child care and tuition assistance for 
training or community college classes under the same priorities and guidelines as Project Independence 
participants. Their eligibility for entitlement benefits in addition to AFDC - such as Food Stamps 
and Medicaid - was unaffected. 

Because the two groups were created by chance, using a lottery-like process, there was only 
one systematic difference between them at the outset of the study: Only those in the program group 
could be involved in Project Independence. As a result, the control group would provide information 
on the levels of employment, earnings, welfare receipt, and welfare payments that the program group 
would have reached if it had not had access to Project Independence and had not been required to 
participate. Therefore, a comparison of the two groups' behavior over time would provide the most 
reliable estimate of the difference that Project Independence's services and mandates made in the 
program group's subsequent labor market and welfare outcomes. These differences are referred to 
as the program's "effects" or "impacts." 

The data used for the full research sample came from automated, statewide earnings and 
welfare payment records. Other oata used in the report include surveys administered 12 months after 
random assignment to a subsample of 916 program group members and, at 24 months after random 
assignment, to subsamples of 520 program group members and 509 control group members; program 
casefile information collected for a subsample of program and control group members at selected 
Project Independence units; and fiscal records provided by HRS and other state agencies. 

The Project Independence evaluation differed in several intended and one unintended way from 
random assignment evaluations of other welfare-to-work programs, and these must be kept in mind 
when comparing results. Each of these differences most likely acted to make the measured impacts 
of Project Independence lower than they otherwise would have been. First, random assignment took 
place at the point when people were either applying for AFDC or being assessed as to their eligibility 
for continuing AFDC receipt. In most other evaluations, random assignment took place at a later 
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point, when individuals showed up for a program orientation. This meant that some members of the 
Project Independence research sample may never have been approved for AFDC or left the roils before 
engaging in even the first program activity, orientation. Second, 88 percent of the Project 
Independence research sample were first-time AFDC applicants or were reapplying for AFDC after 
having been off the rolls, a much higher proportion than in other evaluations. This suggests that the 
Project Independence sample may have been somewhat less "disadvantaged," on average, than other 
samples and may have been more likely to find work and leave welfare on their own, without showing 
any additional effect from the program. 

Finally, during the course of data collection, it was discovered that some control group 
members had been exposed to Project Independence by attending a program orientation or, in a smaller 
number of cases, participating in iis employment-related activities. Although it is impossible to 
estimate precisely how much such exposure may have influenced individual behavior, this departure 
from the research design may have reduced the level of measured impacts by as much as 20 percent. 
However, it is not likely to have affected the pattern of impacts over time or the differences among 
subgroups. 

Participation in Pro|ect Independence by Program Group Members During the First Year of 
Foliow-Up^ 

• Project Independence achieved a high degree of compliance with its imtial 
participation requirements. 

During the first year of follow-up, 77 percent of all program group members attended 
orientation. Approximately half of those who did not attend orientation were referred for an AFDC 
grant reduction (also known as a "sanction") because they had not provided an acceptable reason for 
missing orientation. In all, 98 percent of the program group made contact with Project Independence 
either by attending orientation or, if they did not, by being referred for a sanction or being legitimately 
excused from the participation mandate. 

During the same period, more than half of those who attended orientation participated in the 
program's employment-related activities. In all, 43 percent of all program group members 
(representing 56 percent of those who attended orientation) participated in at least one of the program's 
employment- related activities. These are similar to the rates found in studies of pre-JOBS mandatory 
welfare-to-work initiatives and California's JOBS program (the Greater Avenues for Independence 
Program, or GAIN), the only JOBS program with comparable evaluation data. 

• Reflecting the program's emphasis on immediate entry into employment, 



^This section presents several key findings from MDRC's first report on Project Independence, which 
analyzed the program's implementation by examining program group members' panicipation in the various 
stages of the program model during the first year after they entered the study. See James J. Kemple and 
Joshua Haimson, Florida's Project Independence: Program Implementation, Participation Patterns, and First- 
Year Impacts (MDRC, January 1994). 
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independent job search was the most commonly used activity. For those 
least able to find work on their own, efforts were made to provide 
education and training. 

During the first year of follow-up, 31 percent of the program group members participated in 
independent job search activities. This represents over 70 percent of those who enrolled in at least one 
employment-related activity through Project Independence, Ten percent of the program group 
members participated in a job club, either as their only activity or in combination with other activities. 
This contrasts with other welfare-to-work programs that also emphasized inmiediate job placement for 
large portions of the caseload, but that did so primarily through more intensive and expensive activities 
such as job clubs and supervised individual job search assistance. In addition, 18 percent of the 
program group (representing 41 percent of those who enrolled in Project Independence employment- 
related activities) participated in an education or training activity. 

• Project Independence was only modestly successful in implementing an 
ongoing participation mandate. Participation in program activities was 
usually short. 

During the first year of follow-up, program group members participated in employment-related 
activities for an average of one and a half of the nearly eight months in which they were registered 
for the program, or less than one-quarter of the time. At the end of the first year of follow-up, less 
than one-third of the program group members still registered in Project Independence were engaged 
in employment-related activities. Nearly 40 percent of those still registered (or 13 percent of all 
program group members) were left unaccounted for by Project Independence case managers, and many 
of the rest were waiting to start an activity, in the process of being sanctioned for not meeting the 
participation requirements, or temporarily deferred from participation. 

Participation in All Employment-Related Activities by Program and Control Group Members 
During the Two Years of Follow-Up 

Previous studies of welfare-to-work programs have found that many AFDC applicants and 
recipients enroll in local education or training programs, even when they are not required to do so and 
when they do not have access to case management services provided by programs like Project 
Independence. To determine whether Project Independence increased participation in employment- 
related activities above and beyond what people would have done on their own, the study analyzed the 
participation in such activities by more than 1,0(X) program and control group members over the two 
years following their entry into the study. 

• Sixty-four percent of program group members participated in emplojoment- 
related activities through Project Independence and other sources. 

As shown in Table 1, 64 percent of program group members reported having participated in 
at least one employment-related activity during the two years following their random assignment. 
Eleven percent of program group members reported on the 24-month survey that they first enrolled 
in these activities during the second year of follow-up (not shown in the table). Forty-three percent 
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TABLE 1 



TWO-YEAR IMPACTS OF PROJECT INDEPENDENCE ON RATES OF PARTICIPATION 
IN EMPLOYMENT- RELATED ACTIVITIES 



Activity 



Program 
Group 
(%) 



Control 
Group 

(%) 



Difference 



Any employment-related activity (a) 



63.9 



40.1 



23.8 



Independent job search or job club (a) 
Independent job search 
Job club 



42.7 
41.0 
13.2 



18.7 
17.9 
5.8 



24.0 
23.1 
7.4 



Education or training (a) 
Adult basic education or GED preparation classes 
English as a Second Language instruction 
Vocational training or post-secondary education 
On -the - job training 



41.5 
12.1 

4.6 
29.2 

4.C 



30.5 
8.3 
3.0 

21.4 
3.1 



11.1 
3.9 
1.7 
7.8 
0.9 



NOTES: This table is based on responses to the 24-month survey by 1 ,029 program and control group members. 

The percentages include all those who ever participated in the specified activities during the two 
years of follow-up, regardless of whether they were Project Independence activities. 

Tests of statistical significance of differences between the program and control groups' participation 
rates were not performed. 

(a) Some individuals participated in more than one activity during the follow-up period; therefore, the 
percentages for the specified activities within this category add up to more than the percentage for the category Itself. 
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of the program group participated in independent job search or job club activities, mostly through 
Project Independence. Forty-two percent participated in education or training activities, with less than 
half of this participation having been provided through (or approved by) Project Independence. 

• Forty percent of control group members participated in employment- 
related activities* 

Table 1 shows that 40 percent of control group members reported having participated in at least 
one employment-related activity during the two-year follow-up period. Nineteen percent participated 
in at least one independent job search or job club activity, and 31 percent in at least one education or 
training activity. The higher rate of participation in the latter activities can be attributed, in large part, 
to the relatively wide availability of education and training services in the community. 

• Contrary to the research design, some control group members participated 
in employment-related activities through Project Independence* 

As noted earlier, the research design used for the evaluation specified that control group 
members were not to participate in Project Independence. However, using data collected from Project 
Independence casefiles, it was estimated that nearly 8 percent of control group members attended 
orientation and participated in employment-related activities, mostly independent job search, through 
Project Independence. An additional 12 percent of the control group attended orientation or were 
referred for a sanction for not attending orientation. However, none of these individuals participated 
in the program's employment-related activities. In all, therefore, up to an estimated 20 percent of the 
control group were exposed to some aspect of Project Independence. This departure from the research 
design narrowed the overall participation difference between program and control group members, 
which means that the measured impacts are likely to underestimate the program's true effect. 
Nevertheless, the vast majority of control group members were not required to engage in employment- 
related activities and, as discussed below, there were very large program-control group differences in 
participation. Thus, the experiences of the control group still provide the best benchmark for assessing 
whether Project Independence increased the program group's level of involvement in tliese activities, 
and thereby increased their earnings and decreased their AFDC receipt. 

• Project Independence produced a substantial increase in the use of 
employment-related activities among program group members* Most of 
this increase was accounted for by their much greater use than the control 
group of relatively low-cost independent job search activities* 

Table 1 shows that Project Independence produced a 24 percentage point increase (relative to 
the control group) in the rate at which program group members participated in employment-related 
activities through Project Independence or other agencies in the community. Most of this increase was 
due to an increase in the rates at which program group members participated in independent job 
search, either as their only activity or in combination with education and training activities. The 
program produced a more modest increase in the use of education and training services. 

• For women with no preschool-age children, Project Independence's impact 
on participation rates was similar for those who entered the study early in 
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the random assignment period or later on. For women with preschool-age 
children, however, the program's impact on participation was much 
smaller for the late group than for the early group. 

Table 2 presents the program-control group differences in (i.e., the program's impact on) 
participation in employment-related activities. It shows that Project Independence's impact on overall 
participation rates for those whose youngest child was age six or older was just over 20 percentage 
points for both the early and late groups. For those with preschool-age children, however, the impact 
declined from 35 percentage points for the early group to 24 percentage points for the late group, 
basically owing to a large decrease in the program's impact on participation in education and training 
activities (from 19 percentage points for the early group to less than 3 percentage points for the late 
group). Program staff have partly attributed the greatly diminished use of education and training by 
mothers of young children to the sharply reduced availability of child care, especially for participants 
in longer-term activities such as education and training, (The curtailment of child care probably 
affected those with young children the most,) In addition, hiring freezes and increasing caseloads 
prevented case managers from providing a consistent level of case management. These important 
changes, which began in January 1991, are fundamental to the analysis in this report (as discussed in 
the section on the evolution of Project Independence) and help explain the change over time in 
participation patterns among women with preschool-age children. 

For the Full Sample: The Two- Year Impacts of Project Independence on Employment. Earnings^ 
and Welfare Receipt 

Each impact estimate presented below is the difference between the average of an outcome 
(e.g., earnings, AFDC payments, or Food Stamp payments) for ^2// program group members and the 
average of the same outcome for all control group members. As a consequence, average earnings 
necessarily include zero dollar amounts for sample members who were not employed during the period 
in question. Similarly, average AFDC or Food Stamp payments include zero dollar amounts for 
sample members who did not receive AFDC or Food Stamps, respectively, during the period in 
question. (Asterisks following an impact estimate in the tables mean that the impact was statistically 
significant - in other words, that one can be more confident that the difference was the result of 
Project Independence and not statistical chance.) 

• For the full sample, Project Independence produced a modest decrease in 
program group members' AFDC and Food Stamp receipt - a decrease 
that persisted over the two-year follow-up periods 

Table 3 shows that Project Independence produced a decrease in AFDC payments averaging 
$265 per program group member over the two-year follow-up period (a 6.2 percent savings relative 
to the control group average). There was a slight decline from $152 in the first year to $113 in the 
second year, but the trend in impacts on AFDC payments during the second year of follow-up (not 
shown in the table) suggests additional savings in the future. Project Independence also produced a 
reduction in Food Stamp payments averaging $136 per program group member over the two-year 
period. 
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TABLE 2 



TWO-YEAR IMPACTS OF PROJECT INDEPENDENCE ON RATES OF PARTICIPATION 
IN EMPLOYMENT-RELATED ACTIVITIES FOR SUBGROUPS DEFINED BY 
AGE OF YOUNGEST CHILD AND EARLY OR LATE ENTRY INTO THE RESEARCH SAMPLE 



Subgroup and Activity 



Youngest child, age 6 or older 

Early group 

Any employment- related activity 
Independent job search or job club 
Education or training 



Program 
Group 
(%L 



65.6 
45.4 
44.5 



Control 
Group 
(%) 



45.0 
23.0 
33.0 



Difference 



20.6 
22.4 
11.5 



Late group 

Any employment-related activity 
Independent job search or job club 
Education or training 



60.0 
41.9 
40.6 



39.6 
21.5 
29.5 



20.4 
20.4 
11.1 



Y oungest child, age 3-5 



Early group 

Any employment- related activity 
Independent job search or job club 
Education or training 



77.8 
50.0 
50.0 



42.4 
20.7 
31.5 



35.4 
29.4 
18.5 



Late group 

Any employment-related activity 
Independent job search or job club 
Education or training 



60.2 
41.7 
33.3 



35.8 
11.2 
30.6 



24.4 
30.5 
2.7 



NOTES: The percentages include all those who ever participated in the specified activities during the two 
years of follow-up, regardless of whether they were Project Independence activities. 

Tests of statistical significance of differences between the program and control groups' participation 
rates were not performed. 
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TABLE 3 



TWO-YEAR IMPACTS OF PROJECT INDEPENDENCE ON EMPLOYMENT, EARNINGS, 
AFDC RECEIPT, AND FOOD STAMP RECEIPT FOR THE FULL SAMPLE 



Outcome and Follow-up Period 



Program 
Group 



Control 
Group 



Difference 



Percentage 
Change 



Ever employed (%) 
Years 1 -2 

Yean 
Year 2 

Last quarter of year 2 



66.3 

55.4 
53.3 

38.3 



63.8 

52.6 
51.2 

37.8 



2.5 *** 

2.8 *** 
2 -J *** 

0.4 



4.0% 

5.4% 
4.1% 

1.1% 



Average total earnings ($) 
Years 1 -2 

Year 1 
Year 2 



5766 

2548 
3219 



5539 

2401 
3138 



227 * 

146 ** 

80 



4.1% 

6.1% 
2.6?/o 



Ever received any AFDC payments (%) 
Years 1 -2 

l^st quarter of year 1 
Last quarter of year 2 



88.3 

64.6 
51.2 



89.5 

68.7 
53.6 



-1.1 ** 

—4.2 *** 
—2.4 *** 



-1.3% 

-6.0% 
-4,5%- 



Average total AFDC payments received ($) 
Years 1-2 

Year 1 
Year 2 



4028 

2196 
1832 



4293 

2348 
1945 



-265 *** 

-152 *** 
-113 *** 



-6.2% 

-6.5% 
-5.8% 



Ever received any Food Stamps (%) 
Years 1 -2 

Last quarter of year 1 
Last quarter of year 2 



90.5 

70.5 
56.8 



91.0 

71.8 
57.4 



-0.5 

-1.3 
-0.6 



-0.6% 

-1.8% 
-1.0% 



Average value of Food Stamps ($) 
Years 1 -2 

Year 1 
Year 2 



4002 

2148 
185S 



4138 

2213 
1925 



-136 *** 

-66 *** 
-70 *** 



-3.3% 

-3.0% 
-3.6% 



NOTES: Dollar averages include zero values for sample members who were not employed or were not receiving 
AFDC or Food Stamps. 

A two -tailed t-test was applied to the differences between program and control groups. Statistical 
significance levels are indicated as *** = 1 percent; ** = 5 percent; * = 10 percent. 
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Although Project Independence reduced the proportion of program group members receiving 
AFDC in the last quarter of year 2 by 2.4 percentage points, slightly more than half of all program 
group members (51 percent) were receiving some AFDC payments at that final point in the two-year 
follow-up. 

• For the fuU sample, the modest earnings gains achieved by program group 
members in the first year of follow-up declined greatly in the second year. 
Overall, Project Independence produced only a small increase in the 
average eanungs of program group members over the two years. 

Table 3 shows that, over the two-year follow-up period, Project Independence produced an 
increase of $227 in the average earnings of program group members (a 4. 1 percent increase relative 
to the control group average). Earnings effects declined from $146 in the first year to a statistically 
insignificant $80 in the second, and it is likely that any earnings impacts for the full sample in future 
years will be smaller than those for the first two years. The decline in earnings gains during the 
second year of follow-up for the full sample is unusual, since evaluations of most other welfare-to- 
work programs found at least modest and statistically significant earnings gains through three years 
of follow-up and even beyond. 

Part of the explanation for these small, short-lived earnings gains is differences in the pattern 
of effects for subgroups of the full research sample (to be discussed shortly). The characteristics of 
the jobs program group members obtained is also part of the explanation. Findings from the 
evaluation's survey of a subsample of program and control group members indicate that Project 
Independence produced an increase in employment that was mainly in part-time, relatively low-paying 
jobs. The survey findings show that employed program group members reported hourly wages, weekly 
hours, and weekly earnings slightly below; those of employed control group members. Also, these jobs 
do not seem to have lasted as long as jobs held by control group members. It appears, however, based 
on the experience of the control group, that the program group members who took these jobs would 
not have worked at all in the absence of the program. 

The Two«Year Impacts of Different Versions of Project Independence for Subgroups 
With and Without Preschool-Age Children 

The full-sample findings discussed above mask important differences in the effectiveness of 
Project Independence for key subgroups within the welfare population. The Project Independence 
evaluation was designed specifically to determine the program's effectiveness for AFDC applicants and 
recipients with preschool-age children, as well as for those with no preschool-age children. (Prior to 
JOBS, the latter was the traditional mandatory group for welfare-to-work programs, and therefore the 
subject of most evaluations.) This question became even more salient over the course of the evaluation 
as child care could no longer be offered to most program participants and other program resources 
were stretched to cover an expanding caseload. As noted above, women with preschool-age children 
were particularly likely to have been affected by the reduction in child care and other resources to 
support their participation in longer-term activities such as education and training. To explore these 
issues, impacts were first estimated separately for the early group of sample members (who were 
referred to Project Independence before January 1991) and for the later group (who were referred to 
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the program between January and August 1991). Then, witliin these two groups, impacts were 
estimated separately for women with preschool-age children and for women with no preschool-age 
children. It should be noted that the smaller sample sizes of these subgroups make the impact 
estimates less precise than those for the full sample. 

• Larger and more persistent program effects were found for sample 
members referred to Project Independence before program resom'ces were 
strained by growing AFDC caseloads. 

The top section of Table 4 shows that Project Independence produced two-year earnings 
impacts of $439 for individuals who entered the research sample before January 1991, a period when 
Project Independence was operated closer to the model intended by HRS. This impact was more than 
four times the $99 earnings impact for those who entered later (although the difference between the 
early and late groups was not statistically significant). During that later period, as discussed below, 
the program spent less money per participant, but it sought to continue providing at least minimal 
services to as many participants as possible. Importantly, earnings impacts for the early group did not 
decline over time. In fact, they grew steadily from $216 in year 1 to $223 in year 2 to $277 in year 
3. In all, for the early group. Project Independence produced an average earnings increase of $716 
per program group member over three years. Earnings impacts for the late group were not only 
smaller to begin with, but also declined to about zero by the middle of the second year of follow-up 
(not shown in the table). 

The difference in earnings impacts between the early and late groups was owed, in large part, 
to the early group's greater employment stability. Those from the early group who found jobs were 
more likely to remain employed, leading to a greater impact on total time employed and, hence, to the 
greater impact on total earnings. 

Unlike impacts on earnings, impacts on AFDC payments were similar for the early and late 
groups. AFDC payment impacts declined somewhat in the second year for both groups, although they 
remained statistically significant. 

• Project Independence produced AFDC savings both for sample members 
with preschool-age children and for those who only had older children. 

The bottom section of Table 4 shows that, over the two-year follow-up period. Project 
Independence produced an average of $304 in AFDC savings for those whose youngest child was age 
six or older at the time of random assignment, and $210 for those whose youngest child was between 
the ages of three and five (i.e., preschool-age). The table also shows that Project Independence 
achieved two-year AFDC savings for each of these subgroups within both the early and late groups. 
Those with preschool-age children, however, showed less persistent AFDC effects. AFDC impacts 
for the early group with preschool-age children grew from year 1 to year 2 but then declined to about 
zero by year 3. AFDC impacts for the late group with preschool-age children declined significantly 
after only the first year. In contrast, AFDC savings for those with no preschool-age children declined 
much more gradually from year 1 to year 2, and for the early group, they declined hardly at all from 
year 2 to year 3. 
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TABLE 4 



m 

CO 



TWO-YEAR IMPACTS OF PROJECT INDEPENDENCE ON EARNINGS AND AFDC PAYMENTS 
FOR SUBGROUPS DEFINED BY EARLY OR LATE ENTRY INTO THE RESEARCH SAMPLE AND AGE OF YOUNGEST CHILD 



Subgroup 
Early group 
Late group 

Youngest child, 
age 6 or older 

Early group 

Late group 

Both groups 

Youngestchild, 
age 3-5 

Early group 

Late group 

Both groups 



Impact on Average Total Earnin gs i$) 

Total 

_ . Years 1 -2 Year l _ Year 2 Years (a)^ 



433 
99 



489 
487 * 
473 ** 



358 
-175 
20 



216 ** 
111 



278 * 
307 ** 
284 *** 



142 
14 
54 



223 * 
-12 



211 
181 
190 



216 
-189 
-34 



277 * 
n/a 



424 ** 
n/a 
n/a 



97 
n/a 
n/a 



Impact on Average Tota l AFDC Payments ($) 



Total 
Years 1 -2 



—241 *** 
->289 *** 



-244 ** 
-347 *** 
-304 *** 



-203 * 
-241 *** 
-210 *** 



Yean 



-125 *** 
-173 *** 



-139 *** 
-192 *** 
•171 *** 



-86 

-182 *** 
-133 *** 



Year 2 



-115 *** 
-116 *** 



-105 * 
-155 *** 
-133 *** 



-117 
-59 
-77 



Year 3 



-65 

n/a 



•101 
n/a 
n/a 



-2 
n/a 
n/a 



NOTES: Dollar averages include zero values for sample members who were not employed or were not receiving AFDC. 

A two-tailed t-test was applied to the differences between program and control groups. Statistical significance levels are indicated 
as *** = 1 percent; ** = 5 percent; * = 10 percent. Differences in impacts across the subgroup categories were not statistically significant at the 10 percent level. 

(a) The third-year earnings figures for the early group were calculated as 4/3 the total for the first three quarters of that year. Third -year data were 
not available (n/a) for the late group. 
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• Earnings impacts were found mainly for individuals with no preschool-age 
children. Among individuals with preschool-*age children, only those in the 
early group, who were referred to Project Independence when child care 
and case management resources were more readily available, showed 
evidence of achieving short-*term earnings gains. 

The patterns of earnings impacts were quite different depending on whether families included 
children between the ages of three and five. For those with no preschool-age children. Project 
Independence produced two-year earnings gains averaging $473 per program group member, with the 
gains being similar for the early and late groups. Although, for both the early and late groups, impacts 
declined from year 1 to year 2, they rebounded to $424 for the third year of follow-up for the early 
group, suggesting that their earnings impacts may persist beyond the period for which evaluation data 
were collected. 

Earning i impacts for those with preschool-age children were weaker, especially for the late 
group, which showed virtually no earnings impact in year 1 and then a negative effect in year 2. The 
early group with preschool-age children showed a (statistically insignificant) increase in earnings gains 
from year 1 to year 2, but this declined in year 3. As discussed above, a key difference between the 
early and late groups with preschool-age children was the dramatic decline in the use of education and 
training services by the late group. Overall, these findings provide evidence of the challenge of 
achieving lasting effects for women with preschool-age children, a challenge that was made more 
difficult in the face of limited child care and other resources. 



Two- Year Impacts on Subgroups Defined by Job-Readiness Status and Welfare History 

• For sample members the program designated "not job-ready" and targeted 
for education and training services, Project Independence produced 
delayed earnings impacts, which emerged in the second year of follow-up. 

Project Independence targeted education and training services to a particular group whose 
education level and paucity of prior work experience might be barriers to getting a job. The pattern 
of earnings impacts found for this "not job-ready" subgroup is consistent with a widely held view that, 
if human capital development activities (i.e., education and training) increase earning power overtime, 
they do so after some delay. As shown in the top section of Table 5, those designated as not job-ready 
had no earnings gain in year 1, but their gains in year 2 were much larger ($290 per program group 
member). AFDC impacts for the not job-ready were weak, however, and did not grow over time. 
In contrast, the job-ready subgroup achieved modest and persistent AFDC savings of $317 over two 
years. However, earnings gains for this group declined from $195 in year 1 to a statistically 
insignificant $92 in year 2. 

• For sample members who had received AFDC for a total of two years or 
more prior to being referred to the program, Project Independence 
produced larger-than-average earnings gains and AFDC savings. 

The bottom section of Table 5 shows that Project Independence produced statistically significant 
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TABLE 5 



TWO-YEAR IMPACTS OF PROJECT INDEPENDENCE ON EARNINGS AND AFDC PAYMENTS 
FOR SUBGROUPS DEFINED BY JOB-READINESS STATUS AND WELFARE HISTORY 







Average Total Earnings 






Average Total AFDC Payments 




Subgroup and Follow-up Peripd 


Program 
Group 


Control 
Group 


Difference ($) 


Percentage 
Chanqe (%) 


Program 
Group ($) 


Group {$) 


Difference ($) 


Percentage 
Change (%) 


Job- readiness status (a) 


















Job -ready 
Years 1-2 


6250 


5963 


287 * 


4.8% 


3924 


4241 


-317 *** 


-7,5% 


Year 1 
Year 2 


2770 
3480 


2575 
3388 


195 *** 
92 


7.6% 
2.7% 


2141 
1783 


2314 
1927 


-173 *** 
—1 44 *** 


-7.5% 
-7,5% 


Not job -ready 
Years 1 -2 


2435 


2117 


318 


15.0% 


4518 


4662 


-144 


-3,1% 


Year 1 
Year 2 


1035 
1400 


1008 
1109 


27 
290 * 


2.7% 
26.2% 


2467 
2051 


2564 
2098 


-97 
-47 


-3,8% 
-2.2% 


Welfare hestory (b) 


















First -time applicant 
Years 1 -2 


6734 


6680 


54 


0.8% 


3286 


3538 


-252 *** 


-7,1% 


Year 1 

Year ? 


2959 
3775 


2920 
3759 


39 
15 


1.3% 
0.4% 


1830 
1456 


1972 
1566 


-.'j4'j 
-110 *** 


-7.2% 
-7,1% 


1 ess than two years (c) 
Years 1 -2 


5651 


5564 


87 


1.6% 


4188 




— ^yo 


-6.6% 


Year 1 
Year 2 


2516 
3134 


2367 
3197 


150 

-63 


6.3% 
-2.0% 


2274 
1914 


2474 
2010 


-200 *** 
-96 * 


-6,1% 
-4,8% 


Two years or more (c) 
Years 1 - 2 


4382 


3644 


738 *** 


20.3% 


4946 


5353 


-407 *** 


-7,6% 


Year 1 
Year 2 


1969 
2413 


1584 
2060 


385 *** 
353 *** 


24.3% 
17.1% 


2642 
2304 


2831 
2522 


-189 *** 
.-218 *** 


-6.7% 
-8,6% 



NOTF n Dollar averages include zero values for sample members who were not employed or were not receiving AFDC. 

A two-tailed t-test was applied to the differences between program and control groups. Statistical siglnificance levels are indicated as *** = 1 percent; 
^ 5 percent. * = 10 percent. 

(a) Sample members were defined as "job-ready" if they had completed at least 1 0th grade or were employed for at least 1 2 of the 36 months prior to random 
a'..signment They were defined as "not job-ready" if they did not meet '•'oth criteria. These definitions are based on those used by Project Independence during the 
f.Tidnm assignment period. 

(b) Welfare history refers to the total number of months accumulated from one or more spells on an individual's or spouse's AFDC case. It does not 
'fiL-iude AFDC receipt under a parent's name. . ^ 

(c) This subgroup includes both current recipients and applicants with prior AFDC spells. X 
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reductions in two-year AFDC payments for all three groups defined by the accumulated time they spent 
on AFDC (as adults) prior to being referred to Project Independence: first-time AFDC applicants 
($252), and applicants and recipients who had received AFDC for less than two years ($295) or at 
least two years ($407). Despite the consistent AFDC savings for all three groups, the program 
produced statistically significant earnings gains only for those who had previously received AFDC for 
two years or more — a somewhat more dependent group than the other two. For them, the two-year 
earnings gain was substantial: $738, which represents a 20 percent increase over the control group 
average. The effectiveness of Project Independence for long-term AFDC recipients is important 
because JOBS targets such individuals for priority in services. 



The Cost of Project Independence 

In this study, the cost estimates are reported as averages per sample member. Several different 
kinds of cost estimates have been calculated. First is the total Project Independence cost per program 
group member. This amount included costs incurred by both HRS and the other agencies that provided 
services to which Project Independence participants were referred by the program. Total cost includes 
both the total Project Independence cost and the value of employment-related services received outside 
the purview of Project Independence (e.g., after leaving welfare). A similar total cost was calculated 
for services received by members of the control group. The difference between the average total cost 
for program and control group members is the net program cost per sample member. Net cost was 
the cost measure utilized for the benefit-cost analysis presented in the next section. All costs were 
calculated for a two-year observation period and then projected over an additional three-year period. 
The five-year time horizon for the cost analysis is consistent with that used in other MDRC evaluations 
of welfare-to-work programs. All amounts are expressed in 1993 dollars. 

• The total five-year cost of Project Independence was estimated to be $1,304 
per program group member, making the program a relatively low-cost 
JOBS program. 

By emphasizing independent job search for the majority of its participants and reserving more 
costly education and training services for the "not job-ready," the program adopted a relatively low- 
cost strategy for engaging a very large portion of the AFDC caseload for at least a brief spell in the 
program. Part A of Table 6 shows that an average of $1,304 per program group member was spent 
by HRS and its collaborating agencies to provide Project Independence services. Of this amount, 
about half came from HRS and half from other agencies. Among the costs incurred directly by HRS 
were those for case management, orientation, assessment, child care, and other support services, as 
well as the cost of independent job search and some of the costs of education and training. Most of 
the costs of education and training accrued to the budgets of non-HRS providers (e.g., adult schools 
and community colleges). The proportion of total Project Independence costs attributable to the 
various program components is shown in part B of Table 6. The largest share of resources was spent 
on vocational training and post-secondary (usually community college) education, even though these 
services were used by a relatively small proportion of the caseload. Child care costs accounted for 
more than two-thirds of the support services provided by Project Independence and approximately 8 
percent of all Project Independence costs. 
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TABLE 6 



ESTIMATED PROJECT INDEPENDENCE AND NET COST PER PROGRAM GROUP MEMBER 
WITHIN FIVE YEARS AFTER RANDOM ASSIGNMENT (IN 1993 DOLLARS) 



A. Total Project Independence and non-Project Independence Costs 

Five-Year Average Cost ($) 



Cost Category 



Per Program 
Group Member 



Per Control 
Group Member 



Difference (Net Cost 
per Program Group Member) 



Total Project independence cost 
HPS cost 
Non-HRS cost 

Total non-Project Independence cost 
Total cost 



1304 
550 
753 

1792 
3096 



255 
148 
108 

1691 
1946 



1049 
402 
645 

101 
1150 



B. Percentage Distribution of the Estimated Total Rve-Year Cost of Project Independence, by Component 



Other Support Services 
Orientation and Assessments 3.8% 




Total five-year cost of Project Independence (including expenditures 
by HPS and other agencies): $1 ,304 per program group member 

NOTES: Case management, which accounts for a large share of the HPS costs, is included in the costs for the 
individual components. The percentages in the pie chart do not add to exactly 100 percent because of rounding. 
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• The net five-year cost of providing Project Independence and non-Project 
Independence services to program group members was $1450* 

Overall, it cost an estimated $3,096 per program group member, over a five-year period, to 
provide employment- related services through Project Independence, its collaborating partners, and 
other agencies in the community not affiliated with the program (see part A of Table 6), This amount 
included the cost of activities program group members participated in on their own initiative, without 
being assigned there by Project Independence or having those activities approved by the program. 
Control group members made use of similar services at a cost of $1,946 per control group member 
over the same five-year period. Control group members received services mostly on their own 
initiative, outside Project Independence, As explained earlier, however, they received some services 
through Project Independence. These costs were included in the total cost per control group member. 

Thus, the net cost to government budgets of Project Independence and non-Project 
Independence services was $1,150 per program group member over a five-year period. This net cost 
was the additional investment taxpayers made to provide services for the program group above and 
beyond the cost of services obtained by control group members. 

An Analysis of Project Independence's Benefits and Costs 

The benefit-cost analysis addressed three questions: Were program group members better or 
worse off financially as a result of Project Independence? Was the government's net investment paid 
back by budgetary savings? Did the combined results for program group members and government 
budgets (which economists call the results for "society" as a whole) represent a total net gain or loss? 
In answering these questions, the benefit-cost analysis weighed the effects of Project Independence on 
earnings (and the associated fringe benefits), AFDC payments. Food Stamps, Unemployment Insurance 
benefits, taxes. Medicaid, administrative costs of AFDC and other government welfare programs, and 
the net cost of employment-related services. It did not formally consider intangible positive or 
negative effects of the program, such as the effect of reduced time at home for working parents and 
increased stress, or the value of increased self-esteem and positive role models for children. The 
analysis also assumed that program group members who became employed as a result of Project 
Independence did not take jobs away from other workers (because such "displacement effects" could 
not be measured). All benefits and costs were calculated for the wo to diree years for which data 
were available and then projected over an additi' -nal two to three years, for a total time frame of five 
years. All amounts are expressed in 1993 dollars. 

• Overall, program group members incurred a small net financial loss from 
Project Independence. 

As shown in the "welfare sample" column of Table 7, program group members experienced, 
on average, a net financial loss of $369 over the five-year period used for the benefit-cost analysis. 
This was because the increase in earnings (and associated fringe benefits) produced by the program 
was more than offset by decreases in AFDC, Food Stamps, Unemployment Insurance benefits, and 
Medicaid. The likely explanation is that program group members were probably more apt to leave 
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TABLE 7 



ESTIMATED NET GAINS AND LOSSES FROM PROJECT INDEPENDENCE 
AND RETURN PER PROGRAM GROUP MEMBER 
WITHIN FIVE YEARS AFTER RANDOM ASSIGNMENT, 
BY ACCOUNTING PERSPECTIVE (IN 1993 DOLLARS) 



Accounting Perspective 



Benefit-Cost Component 



Welfare Sample ($) Government Budget ($) 



Society ($) 



Total from earnings 
and fringe benefits 

Total from tax payments (a) 



689 
-21 



0 

67 



689 
0 



Total from transfer payments 
and administrative costs (b) 

Net cost of Project independence and 
non-Project Independence 
activities and services 



-1037 



1155 



■1150 



118 



-1150 



Net gain or loss 
(net present value) 



-369 



72 



-343 



Return to government budget per 
net dollar invested in Project 
Independence and non-Project 
Independence activities and services (c) 



n/a 



1.06 per$1 



n/a 



NOTES: "N/a" means not applicable. 

(a) Includes payroll, income, and sales taxes Florida does not have a state income tax. 

(b) Includes AFDC payments. Food Stamps, Unemployment Insurance benefits. Medicaid, and the 
administrative costs of AFDC and the other government welfare programs. 

(c) This figure was computed by dividing tax payments and transfer program savings by the net cost of 
Project Independence and non -Project Independence activities and services. 
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AFDC for a short time to take a relatively low-wage job, but then not return to the welfare rolls 
immediately after leaving that job. 

• For government budgets, Project Independence produced savings in 
M'elfare payments and increases in tax revenues that slightly exceeded the 
cost of services for program group members. 

Project Independence was intended to reduce the cost of public assistance, and it did. In fact, 
the program recouped all its net costs through reductions in welfare and increases in tax payments 
(because of the earnings gains it generated). As shown in the "government budget" column of Table 
7, Project Independence actually produced a small net gain to government budgets ($72 per program 
group member). One can also consider the cost-effectiveness of Project Independence from the 
standpoint of government budgets by estimating the value of budgetary savings and tax revenue 
increases per dollar of investment (i.e., per dollar of net costs). This measure is called the return to 
budget per net dollar invested. As shown in the second colunm of Table 7, Project Independence 
produced $1.06 in savings and revenues for every additional dollar spent on program group members 
(beyond the control group average expenditure). In other words, the financial returns were large 
enough to allow government budgets to do slightly better than break even. 

The last colimm of Table 7 gives the benefit-cost results from the combined perspectives of 
program group members and government budgets. Referred to by economists as the financial gains 
or losses for "society," this perspective is approximated by summing the welfare sample and 
government budget columns (including only employees' share of payroll taxes). Benefits accrue to 
society through the additional goods and services produced on the job by program participants who 
work. The value of that additional output is represented by the earnings gains of program group 
members. Society also benefits from reduced expenditures in administering welfare programs, which 
occurs through the program's impact on reducing receipt of AFDC and other "transfer payments " (i.e. , 
Food Stamps, Unemployment Insurance benefits, and Medicaid). In this kind of analysis, the 
program's net costs are categorized as a loss from the perspective of society. All other effects (e.g., 
reduced transfer payments and increased tax receipts) represent gains to one group in society and losses 
to another, resulting in no effect for society as a whole. Considering only financial gains and losses, 
the net present value (i.e., the net gain or loss) of Project Independence to society as a whole was 
minus $343 per program group member over five years. This was because the additional earnings that 
accrued to program group members were more than offset by the government's cost of providing 
services to them. 

• From a benefit-cost standpoint, Project Independence was most successful 
for the early group of AFDC applicants and recipients with no preschool- 
age children. For this group, the program produced budgetary savings 
and made program group members better o^ financially. 

Table 8 presents a sunmiary of the benefit-cost findings for the sample members with and 
without preschool-age children, within the early and late groups. Because of small sample sizes and 
data limitations, these findings are less precise for the subgroups than they are for the full sample. 
The first row of the table shows that the early group with no preschool-age children achieved a small 
net financial gain of $333 over five years. At the same time, the program produced an additional $621 



ES-26 



TABLE 8 



BENEFIT-COST RESULTS OF PROJECT INDEPENDENCE 
FOR SUBGROUPS DEFINED BY AGE OF YOUNGEST CHILD AND 
EARLY OR LATE ENTRY INTO THE RESEARCH SAMPLE (IN 1993 DOLLARS) 



cn 



Subgroup 

Youngest child^ age 6 or p[der 
Early group 
Late group 

Youngest child, age 3-5 
Early group 
Late group 



Net Cost of Project 
Independence and 
non-Project Independence 
Services ($) 



1128 
877 



Estimated Net Gain or Loss (Net Present Value) per Program Group Member 
Within Five Years After Random Assignment, by Accountin g Perspective ($) 



1831 
894 



Welfare 



333 
-755 



277 
-748 



Government 
Budget ($) 



621 
745 

-1220 
-402 



Society ($) 



834 
-53 

-995 
-1130 



Return to 
Government Budget 
per Net Dollar 
Invested (a) ($ ) 



1 .55 per $1 
1 .85 per $1 

0.33 per$1 
0.55 per $1 



NOTE: (a) This figure was computed by dividing tax payments and transfer program savings by the net cost of Project Independence and 
non-Project Independence activities and sen/Ices. 
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in welfare savings and tax revenues for government budgets, resulting in a return of $1.55 for each 
additional dollar invested in employment-related activities for program group members (i.e., beyond 
the investment in control group members). 

In contrast, the late group of sample members with no preschool-age children did not accrue 
enough additional earnings to offset the resulting reduction in welfare payments. The second row of 
the table shows that this resulted in a net financial loss of $755, on average, for this group. However, 
the government's smaller net investment in the late group ($877 compared to $1,128 for the early 
group with no preschool-age children) still produced welfare savings resulting in a net gain to 
government budgets of $745 (or $1.85 for each additional dollar invested). 

The bottom section of Table 8 shows the benefit-cost results for those with preschool-age 
children, within the early and late groups. Those in the early group came out slightly ahead 
financially, while those in the late group incurred a financial loss of $748. Government budgets did 
not break even for either group. 

A Comparison with the Results of Evaluations of Other Welfare-to-Work Programs 

• Project Independence's impacts on AFDC payments were in the middle range of 
the e^iects produced by large-scale, pre-JOBS welfare-to-work programs that 
provided mainly job search activities. Its earnings impacts were smaller, except 
for the early group without preschool-age children. 

Major differences in research designs (e.g., the early point of random assignment in this 
evaluation), research samples (e.g., the great preponderance of welfare applicants in the present 
sample), program activities (e.g., the unusually heavy emphasis on independent job search), and local 
environments (e.g., dramatically rising caseloads and stretched resources) make comparisons of Project 
Independence's effects to those of other programs difficult to interpret. Nonetheless, comparing 
different programs' general patterns of effects can provide a useful perspective from which to judge 
the results of Project Independence. 

The very low-cost, mostly job search programs of the early 1980s - the kinds of programs 
JOBS was designed to replace - had a range of effects on AFDC payments, with some producing no 
appreciable welfare savings. Project Independence's 6 percent reduction in two-year AFDC payments 
falls roughly in the middle of the observed range of welfare savings. At the same time. Project 
Independence was virtually unique in showing a decline in impacts on earnings after the first year of 
follow-up. However, its more promising results for the early group who had no preschool-age 
children rank in the mid-range of earlier programs' earnings impacts. 

• Compared to the overall findings for California's JOBS program, Project 
Independence produced shnilar percentage reductions in AFDC payments. 
How the earnings gains compare depends on the subgroups beutig 
considered. 

Findings from a random assignment evaluation of California's Greater Avenues for 
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Independence (GAIN) Program, the nation's largest JOBS program, provide another point of 
comparison for assessing the Project Independence results. The GAIN program differed from Project 
Independence in a number of ways. For example, GAIN placed much greater emphasis on basic 
education as a first activity for the large group of enxoUees who were deemed to need it (well over half 
the research sample). Moreover, it was a more expensive program (see Table 9), for reasons that 
included its greater emphasis on basic education, its much greater reliance on job clubs rather than 
unsupervised, independent job search, and its assignment of considerably smaller caseloads to GAIN 
case managers. 

In comparing the Project Independence and GAIN findings, it is also important to recognize 
that California's AFDC system, in contrast to Florida's, is a state-supervised but county-administered 
system, which led to greater county variation in program practices than was observed in Florida. For 
the latter reason, the GAIN evaluation focused much more on county-specific impacts, benefits, and 
costs than on the six-county average results. (County-specific impacts for the key subgroups were not 
estimated for the Florida study.) Importantly, the GAIN analysis found that one county (Riverside) 
produced the most impressive results ever measured for a large-scale welfare-to-work program. Those 
results have taken on national significance because they point to the potential of a JOBS program to 
achieve substantial earnings gains and welfare savinf for a wide range of welfare recipients, while 
helping government budgets and the welfare sample come out ahead. 

Estimates of net costs and impacts on earnings and AFDC payments for Project Independence 
and GAIN are presented in Table 9.^ The table shows that GAIN (when its cross-county average 
effects are considered) produced bigger average reductions in welfare payments over three years of 
follow-up. However, those larger AFDC savings were partly a function of California's relatively high 
monthly AFDC grant levels. In view of these differences in state grant levels, an alternative approach 
is to focus on the percentage reduction in average AFDC payments made to the program group 
compared to the control group average. Using that measure, the effects of Project Independence were 
actually somewhat larger, as shown in the last column of Table 9. For example, for the early group 
without preschool-age children, the reduction in AFDC payments was 6.3 percent of the control group 
average for years 1 and 2 combined, compared to the 5.6 percent (all-county) figure for GAIN. 

Project Independence's earnings impacts were smaller than GAIN'S. However, it is noteworthy 
that, for the early group in Project Independence whose youngest child was age six or older, the trend 
of increasing earnings gains in the third year of follow-up was similar to the pattern for the GAIN six- 
county average. As for the sample members with preschool-age children, the GAIN results show that 
a JOBS program can produce large earnings gains for this group: an average three-year increase of 
almost $2,000 (i.e., the sum of the annual figures shown in the bottom row of Table 9) across the 
subset of three California counties in which this subgroup was studied. The findings also suggest, 
however, that achieving such results may require more substantial resources than were available to 
Project Independence. 



^To enhance comparability with GAIN, the Project Independence estimates could be adjusted to take into 
account the fact that some control group members were reached by the program, which did not occur in the 
GAIN evaluation. The Project Independence impacts shown in the table are 20 percent smaller than they 
would be if such an adjustment were made. 
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TABLE 9 



PROGRAM IMPACTS FOR SINGLE PARENTS IN PROJECT INDEPENDENCE 
AND THE CALIFORNIA GAIN PROGRAM 



Subgro'jp^and Program , .,„Nel.^ost^)' 
Ypungest chlld, age 6 pr older 



Impact on 

Average Annual E a rnings ($) 

Year 1 Year 2 Year 3 



Impact on 
Average Annual AFDC Payments ($) 

Year 1 Year 2 Year 3 



Percentage Impact 
on AFDC Payments 



Years 1-2 





Project Independence (a) 
Early group (b) 
Late group 


1128 
877 


278 * 
307 ** 


211 

181 


424 ** 

n/a 


-139 *** 
-192 *** 


-105 * 
-155 *** 


-101 * 
n/a 


-6.3% 
-8.4% 




California GAIN (c) 


3422 


266 *** 


512 *** 


636 *** 


-283 *** 


-347 *** 


-331 *** 


-5.6% 


tn 

CO 


Youngest child,. age 3-5 


















O 


Project Independence (a) 
Early group (b) 
Late group 


1831 
894 


142 
14 


216 
-189 


97 
n/a 


-86 
-182 *** 


-117 * 
-59 


-2 
n/a 


-4.3% 
-5.1% 




California GAIN 

Three counties (d) 


n/a 


512 (Z) 


732 (Z) 


71 1 (z) 


-108 (z) 


-194 (Z) 


-350 (z) 


-2.4% 



NOTES: Dollar averages include zero values for sample members who were not employed or were not receiving AFDC. 

A two-tailed t-test was applied to the differences between program and control groups. Statistical significance levels are indicated 
as *** = 1 percent; ** =: 5 percent; * = 10 percent. 

Where results are not available, "n/a" is used. 

(a) Adjustment for the exposure of some control group members to Project Independence could increase estimates by as much as 25 percent. 

(b) The third -year earnings figures foi the early group were calculated as 4/3 the total for the first three quarters of that year. Third -year data 
were not available (n/a) for the late group. 

(c) Estimates are averages across six counties, with results for each county weighted equally. The counties are Alameda. Butte. Los Angeles. 
Riverside. San Diego, and Tulare. 

(d) Estimates are averages across three counties, with results for each county weighted equally. The counties are Alameda, Riverside, and Tulare, 
(z) Statistical tests are not available. 
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• The Project Independence and GAIN benefit-cost results offer some 
evidence of the potential of JOBS to produce net financial gains from both 
the welfare sample and government budget perspectives. 

The full-sample benefit-cost estimates for both programs - which for Project Independence 
show the welfare sample incurring a net loss, and for GAIN show the government budget incurring 
a net loss (not shown in the tables) - seem to suggest that policymakers face a trade-off between the 
goals of saving money for the government and increasing income for poor single parents. However^ 
such a trade-off may not be inevitable. For example, in Project Independence, as previously 
discussed, both the welfare sample and government budgets came out ahead for the early group of 
single parents with no preschool-age children - the group who experienced the program when Project 
Independence resources were less strained. A similar pattern of increased income for the welfare 
sample and budgetary savings was observed for three of the six counties in the GAIN evaluation. But 
although all of these results demonstrate that a trade-off between the goals may not be inevitable, the 
failure to achieve both of them simultaneously for Project Independence's other key groups, especially 
those who did have preschool-age children, and across the combined six counties in the GAIN 
evaluation, is a reminder of the continuing challenge facing JOBS administrators aiming to accomplish 
both objectives. 
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CHAPTER 1 



INTRODUCTION 



This is the second and final report from an evaluation of Florida's Project Independence, which 
the Manpower Demonstration Research Corporation (MDRC) conducted under a contract with 
Florida's Department of Health and Rehabilitative Services (HRS) and with support from the Ford 
Foundation and the U.S. Department of Health and Human Services. It presents findings on the 
effectiveness of Project Independence in moving welfare applicants and recipients into employment and 
off the welfare rolls over a two-year period. It also analyzes whether and to what extent the program 
has been cost-effective. 

Since 1987, when it was created. Project Independence has operated as Florida's statewide 
welfare-to-work program. Its aim is to increase the employment and foster the self-sufficiency of 
applicants for and recipients of Aid to Families with Dependent Children (AFDC), the major 
federal/state cash welfare program. All single-parent AFDC applicants and recipients whose youngest 
child is age three or older are required (i.e., "mandated") to participate in the program unless they 
meet specific exemption criteria.^ Furthermore, they are required to participate continuously for as 
long as they remain on welfare. 

Florida was among a handful of states that anticipated federal welfare reform legislation — the 
Family Suppon Act of 1988 and its centerpiece, the Job Opportunities and Basic Skills Training 
(JOBS) Program - by strengthening the link between receipt of AFDC and obligations, opportunities, 
and suppons for parents in poor families to obtain employment. With minor changes. Project 
Independence became Florida's JOBS program in 1989. Like most states' JOBS programs. Project 
Independence is state-administered and attempts to promote a relatively uniform set of goals and 
operating procedures, while allowing local program offices some discretion to adapt these to their 
particular circumstances. Project Independence operates under a wide range of local circumstances 
across Florida, a state with one of the nation's largest and most diverse AFDC populations. 

MDRC's evaluation of Project Independence began in 1990. It was conducted in nine counties, 
which were selected randomly from among the state's 25 largest in terms of AFDC caseloads and 
which include Florida's eight largest cities as well as some suburban and rural areas. Random 
selection of the counties was important because it enhanced the extent to which the findings can be 
generalized to the state as a whole. The research sample consists of more than 18,000 single parents 
(most of them mothers) in those nine counties. Specifically, it includes those who were applying for 
AFDC or who were being assessed for continuing AFDC eligibility (i.e., their eligibility was being 



^Exempt individuals include those who are age 60 or older, working 30 or more hours per week at the 
mimmuni wage, pregnant and in their second or third trimester of pregnancy, permanently ill or incapacitated, 
or required in the home to care for a physically or mentally impaired household member. Project 
Independence also serves AFDC applicants and recipients who are exempt from the program but who wish 
to volunteer, as well as unemployed parents in two-parent households who are required to participate. This 
report focuses exclusively on single-parent heads of households who were required to prjticipate during the 
period under study; they constituted a majority of the population the program served, 
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"redetermined") between July 1990 and August 1991, and who were required to participate in Project 
Independence (i.e., they were mandatory participants). 

The evaluation is of broad interest because it assesses the effectiveness of a particular JOBS 
approach: one that encouraged immediate entry into the labor force for the majority of its participants 
and reserved most of the program's basic education and occupational skills training resources for those 
least able to find work on their own. The evaluation also provides some of the most extensive 
information to date on the effectiveness of a mandatory JOBS program for single parents with 
preschool-age children. 

During the period of tliis evaluation (July 1990 through September 1993, when the last follow- 
up data were collected), Florida's public assistance programs, like those in most states across the 
nation, experienced considerable stress owing to increasing welfare caseloads, fixed or declining, 
resources being committed to helping those on welfare, and a stagnant or declining economy. The 
report examines whether changes in Project Independence's implementation and environment were 
associated with differences in the program's impact on the earnings and welfare receipt of its 
mandatory participants. It does so by comparing the experiences of people who entered the study 
during its first six months and those who entered subsequently. The first group (designated the "early 
cohort"^ in this report) was referred to Project Independence between July and December 1990, a 
period when Project Independence was operated in a manner more closely matching HRS's original 
model for the program. The second subgroup (the "late cohort") was referred to the program between 
January and August 1991, a period when it was forced to spread a fixed set of resources over a rapidly 
increasing caseload. The findings from this comparison will be particularly relevant to the debate 
about how resource commitments to JOBS programs are likely to influence their effectiveness. 

MDRC's first report on Project Independence focused on analyzing program service delivery 
strategies, participation patterns, and effects on employment, earnings, and AFDC receipt during the 
first year after the individuals in the research sample applied for AFDC or, in the case of current 
recipients, were redetermined to still be eligible for AFDC.^ The report found that Project 
Independence achieved substantial compliance with its participation mandate and produced increases 
in earnings and reductions in welfare payments during the first year after research sample members 
entered the study and were randomly assigned to either a "program group" or a "control group." In 
particular, individuals in the program group (i.e., those who were given access to Project 
Independence and were subject to its participation requirements) earned, on average, 6 percent more 
than those in the control group (i.e., those who were not required or permitted to enter Project 
Independence) and received, on average, C percent less in AFDC payments. Importantly, the earnings 
gains were concentrated among two groups: (1) individuals defined by the program as "job-ready" and 
therefore targeted to participate in an independent job search (rather than basic education or training) 
as their first program activity, and (2) single parents whose youngest child was age six or older. The 
first-year results for the latter group were snnilar to those found for single parents with school-age 
children in studies of a number of pre-JOBS programs and in the evaluation of California's Greater 



^Social scientists often use the term "cohort" when referring to a group of people who experienced a 
particular event during the same time period, in this case entry into the Project Independence research sample. 

■^See James J. Keraple and Joshua Haimson, Florida *s Project Independence: Program Implementation, 
Participation Patterns, and First-Year Impacts (MDRC, 1994), 



Avenues for Independence (GAIN) Program, the most recent evaluation of a JOBS program with 
comparable data. 

The first report emphasized that these early results should be interpreted with caution and that 
a more complete picture of Project Independence's effectiveness should include a longer follow-up 
period and a comparison of the program's benefits and costs. The current report presents longer-term 
(two- to three-year) results and compares the program's costs with its benefits from the perspectives 
of participants, goverronent budgets, taxpayers, and society as a whole. It addresses crucial questions 
about whether the program, with its emphasis on immediate entry into the labor force for most Project 
Independence participants, was able to sustain the short-term earnings gains and welfare savings found 
after the first year of follow-up. It also provides information about whether changes in Project 
Independence during the evaluation period affected its impact on earnings and welfare savings. 

L Project Independence and Changes over Time 

This section of the chapter describes the Project Independence program model. It also 
describes changes in the program and in Florida's AFDC caseload and economy during the evaluation 
period. These changes constitute an important context for interpreting the evaluation findings. 

A. The Project Independence Progi^am Model 

Figure 1 . 1 illustrates the intended flow of AFDC applicants and recipients through the Project 
Independence program model during the evaluation period. For those required to participate, exposure 
to Project Independence began at the HRS Public Assistance Unit (also known in other states as tlie 
Income Maintenance office) when an AFDC applicant or recipient was determined to be mandatory 
for participation. At that time, all such applicants and recipients were informed by the Public 
Assistance Specialist (PAS) that they were required to meet Project Independence participation 
requirements and were scheduled to attend a program orientation appointment within the next 30 days. 
If a person was required to participate but failed to attend orientation and did not provide an acceptable 
reason, a Project Independence case manager was authorized to initiate procedures that could result 
in a reduction of her"* AFDC grant, ^ In some cases, the referral to orientation took place before 
AFDC eligibility was determined. Those who were not approved tor AFDC eligibility, and those who 
fonnd work and left the AFDC rolls before attending an orientation, were no longer required to 
participate in Project Independence, 

During orientation, Project Independence staff provided a more detailed explanation of the 
participation requirements and described the opportunities Project Independence offered to help 



"^Since the overwhelming majority of the single parents in Florida who receive AFDC are women, 
feminine pronouns are used throughout this report. 

^Formal enforcement procedures leading to a grant reduction for failure to comply with Project 
Independence rules are called "sanctioning procedures." Sanctioning procedures can be initiated at any point 
in the program if an individual fails to attend a required appointment or activity. These procedures are 
discontinued when the individual complies with the participation requirement or provides an acceptable reason 
(also known as "good cause") for not participating. 
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FIGURE 1.1 

A SIMPLIFIED DEPICTION OF THE PROJECT INDEPENDENCE PROGRAM MODEL 
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SOURCE: Florida Department of Health and Rehabilitative Services Project Independence Manual. 

NOTES: (a) Basic education activities include high school, adult basic education, GED preparation, and English as a Second Language. 

(b) Post-secondary education and training activities include classroom-based vocational or occupational skills training, on-the-job training, 
customized training, employment preparation, community college, and work experience activities. \jQ 
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registrants get jobs and leave AFDC. It was also during orientation that participants were referred to 
one of two service tracks based on an initial assessment of their educational attainment levels and 
recent work experience.^ During the period when individuals for the research sample were identified 
(July 1990 through August 1991), Project Independence categorized most of its caseload as "job-ready" 
by setting the initial assessment criteria at relatively low levels of education and work experience.^ 
TTiese criteria, combined with an emphasis on case managers' meeting specific job placement 
standards, reflected Project Independence's employment-focused approach to self-sufficiency. 

Those assessed to be "job-ready" were first referred to a two-week individual job search. 
Individual job search activities within Project Independence were usually unsupervised and required 
participants to look for work independently. This contrasts with other welfare-to-work programs that 
have focused on job search assistance and group job club activities as the primary strategies to help 
move participants into jobs as quickly as possible.^ If job search participants were unable to secure 
employment on their own, they were then referred to job club classes, which were designed to teach 
job search skills and to provide supervised opportunities to look for work. 

Figure 1 . 1 shows that Project Independence participants who were assessed as not job-ready 
(and job-ready participants who did not secure employment through job search or job club) were to 
be referred to a more formal assessment of their educational needs and work experiences. During this 
formal assessment, the participant and the case manager were to develop an Employability Plan, based 
on a mutually agreed-upon set of employment goals. These plans usually prescribed education or 
training assignments, but often included job search activities instead of, or in addition to, education 
or training. Participants who remained unemployed after completing their education or training 
assignments — and, usually, an extended independent job search - were, in most instances, reassigned 
to formal assessment to determine additional activities. Some registrants were already in education 
or training activities at the time they attended orientation. These individuals were usually referred for 
formal assessment and were allowed to continue in their present activity if the case manager 
determined that it was appropriate to the participant's employment goal and was consistent with the 
Employability Plan. 

Project Independence was also designed to provide child care, transportation, tools, and 
uniforms participants may have needed to engage in program activities and to secure employment. 



^The program allowed individuals to be temporarily "deferred" from participating if there were legitimate 
reasons. 

^During the period when research sample members were identified, individuals were determined to be 
"job-ready" if they had completed at least the tenth grade or had worked in at least 12 of the previous 36 
months, and to be "not job-ready" if they met neither of these criteria. Although the basic program flow 
model has remained unchanged since Project Independence was first implemented, HRS modified the criteria 
by which individual participants were assessed for initial services. Beginning in October 1991, the job- 
readiness criteria were elevated to include only those who had a high school diploma or a General Educational 
Development (GED) certificate or who had worked in at least 12 of the previous 24 months. These 
modifications enabled a higher percentage of the mandatory caseload to enter education or training activities 
as their initial service. 

^Gueron and Pauly (1991) describes the welfare-to-work programs MDRC evaluated during the 1980s. 
Like Project Independence, these pre-JOBS programs emphasized immediate entry into the labor force for 
most of their participants. However, these programs committed a higher percentage of their resources to job 
search assistance and job club activities than did Project Independence. 
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Figure 1.1 depicts the Project Independence program model as it was intended to operate. 
Chapter 3 describes the program's services (which are noted in the figure) and analyzes the 
participation patterns of those in the research sample who were referred to Project Independence as 
mandatory participants, 

B. Changes in Project Independence and Its Environment Dxiring the Evaluation Period 

Fundamental to understanding the findings in this report are changes that took place in Project 
Independence and its wider economic context over the course of the evaluation. Figure 1.2 displays 
the key milestones for the evaluation, which spanned the period from July 1990 through September 
1993. The bottom of the figure shows the timing of various events related to changes in Project 
Independence and the state during the evaluation period. The figure also illustrates the general trends 
in Florida's AFDC caseload sizes and unemployment rates during the evaluation period. 

The horizontal axis in the figure begins with July 1990, the first month during which AFDC 
applicants and recipients who were mandator}' for Project Independence were enrolled in the study 
(i.e., were randomly assigned to the program or control group),- The sample enrollment period 
(indicated by the shaded areas in Figure 1.2) ended in August 1991, Information about sample 
members' emplojTiient, earnings, and welfare receipt was collected through September 1993. The data 
collection period (also referred to in this report as the "follow-up period") extended from the date an 
individual was enrolled in the study through September 1993. Thus, the report includes two years of 
follow-up information for all sample members and up to three years for those enrolled in July 1990. 

Figure 1.2 shows that between July 1990 and August 1991 - the period during which AFDC 
applicants and recipients were enrolled in the evaluation - Florida's AFDC caseload grew by 25 
percent and the state's unemployment rate increased by 33 percent.^ While the AFDC caseload 
continued to grow through the end of the evaluation period, the state's unemployment rate peaked in 
July 1992 at nearly 9 percent and then fell to approximately 6,8 percent by September 1993. 

Begiiming in early 1991, the dramatic growth of the AFDC caseload was accompanied by a 
series of state hiring freezes, which prevented Project Independence from increasing its staffing 
capacity to meet the growing demand for its services. Also, beginning in January 1991, budget 
constraints forced HRS to restrict the availability of child care for Project Independence participants, 
particularly those who required longer-term services such as education and training. This is significant 
because mandatory registrants with young children could be excused from the participation 



^By contrast, the average monthly caseload for the nation as a whole increased by 19 percent between 
fiscal year 1990 (October 1, 1989, through September 30, 1990) and fiscal year 1991 (October 1, 1990, 
through September 30, 1991). The national unemployment rate increased by 19 percent between July 1990 
and July 1991. AFDC caseload data for Florida were obtained from the Florida Department of Health and 
Rehabilitative Services, District Economic Services, Quarterly Statistical Tables. AFDC caseload data for the 
United States are from the U.S. Congress, House Committee on Ways and Means, Overv/ew of Entitlement 
Programs, 1991, 1992. U.S. and Florida unemployment rates were obtained from the U.S. Department of 
Labor, Bureau of Labor Statistics, 1991, 1992, 1993. 

^^According to MDRC's survey of Project Independence staff, the average client-to-caseworker ratio was 
over 200 by October 1991. (Project Independence does not keep records of these ratios.) 
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FIGURE 1.2 



KEY EVENTS, AFDC CASELOAD SIZES, AND UNEMPLOYMENT RATES 
DURING THE PROJECT INDEPENDENCE EVALUATION PERIOD 
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Key Events: 



7/90 1/91 7/91 1/92 7/92 1/93 7/93 1/94 

Project Independence Evaluation Period: July 1990 through September 1993 

Enrollment of the research sample (i.e., the period of random assignment) occurred from July 1990 through August 1991. 
Follow-up data collection occurred from July 1 990 through September 1 993. 



Changes in job-readiness criteria and 
start of caseload reduction measures 
(October 1991). 



LES assumes administrative 
responsibility for Project Independence 
(July 1992). 



Start of child caie restrictions 
H and HRS hiring freezes 
(January 1991). 

SOURCES: Klorida Department of Health and Rehabilitative Services, District Economic Services, Quarterly Statistical Tables, and 
g J U.S. Dnpartment of Labor, Bureau of Labor Statistics, Current Population Survey. 
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requirements if child care services could not be provided and alternative child care arrangements could 
not be made. The increase in the state's unemployment rate, meant that Project Independence's 
emphasis on moving a high percentage of its participants into the labor market as quickly as possible 
coexisted with a growing shortage of available job openings. It may also account for part of the 
increase in the AFDC caseload as some of those who lost their jobs may have turned to AFDC for 
financial assistance. 

Because of the events that took place during the evaluation period, this report provides what 
amounts to an assessment of two different versions of Project Independence, which can be examined 
by comparing the experiences of early versus later entrants into the research sample. The early cohort 
entered the study between July and December 1990 (as indicated by the lighter of the two shaded areas 
in Figure 1.2) and was exposed to Project Independence when it was operated more as originally 
intended by HRS. While the program was not richly funded, staff were responsible for smaller 
caseloads than they would later handle, child care was widely available, and the labor market was 
relatively robust. Under these conditions. Project Independence staff were better able to enforce the 
ongoing participation mandate and to help participants find work by facilitating their access to 
employment-related activities and support services. The later cohort entered the study between January 
and August 1991 (as indicated by the darker shaded area in Figure 1.2) and was exposed to Project 
Independence at a time when it was forced to spread a fixed set of resources over a rapidly increasing 
caseload. These conditions restricted the staffs capacity to monitor participation and limited the level 
of support they could provide to help participants gain work skills and fmd employment. As noted 
earlier, by comparing the experiences of these two groups, this report will examine whether differences 
in the program's implementation and environment are associated with differences in its effectiveness. 

Late in 1991, after enrollment in the study had ended, two important modifications to the 
program model were implemented.^^ First, HRS changed the job-readiness criteria to enable more 
participants to enroll in education or training, rather than independent job search, as their first activity. 
Second, some counties began implementing new priorities in order to reduce Project Independence 
caseloads. Staff were now allowed to give first preference to those who volunteered for Project 
Independence services, which made them more likely to place a lower priority on enforcing the 
participation requirements for some of those who were determined to be mandatory but had not 
volunteered to participate. 

For several reasons, it is not likely that these modifications had a significant effect on the 
evaluation findings. Both the changes in the job-readiness criteria and the shift to a priority on serving 
volunteers were implemented unevenly over time and across the nine evaluation counties. In some 
places, caseload reduction measures were never formally implemented. Furthermore, most of the 
research sample members had already been referred to Project Independence and had participated in 
a program orientation by the time these changes were implemented. 



^^Also, in June 199 1, the State of Florida began implementing a new computer system to record data 
related to its public assistance programs. Although random assignment in each of the study counties ended 
before this system - called the Florida On-Line Recipient Integrated Data Access System (FLORIDA) - 
came on line in that county, the level of effort devoted to the system's implementation may have affected some 
Project Independence and AFDC program operations. 



Finally, until July 1992, HRS had responsibility for policy development, planning, 
administration, staffing, and operational oversight for Project Independence at the state and district 
levels. In July 1992, HRS subcontracted responsibility for staffing and operating Project Independence 
at the district level to the Department of Labor and Employment Security (LES). Local Project 
Independence staff remained in place under LES and continued to provide case management; monitor 
participation in independent job search and job club activities; and- develop linkages with other state 
agencies, local education authorities, community-based organizations. Job Training Partnership Act 
(JTPA) programs, and local employers, which provide most of Project Independence's employment, 
education, and training activities. HRS retained overall administrative responsibility for the program 
and serves as its primary policymaking agency. The change in departmental responsibilities occurred 
a year after the research sample was enrolled and is not likely to have affected the evaluation results. 



IL The Project Independence Evaluation Design 

The research design for the Project Independence evaluation was conceived with four broad 
objectives: (1) to produce findings that can be generalized to the state's full JOBS-mandatory AFDC 
caseload of single parents; (2) to reliably measure Project Independence's effects on employment, 
earnings, and AFDC receipt compared to what would have been the case if individuals had not had 
access to the program; (3) to capture program effects that accrued from any knowledge of or 
involvement with Project Independence beginning with an individual's AFDC application or 
redetermination; and (4) to measure the program's effects for key subgroups of JOBS-mandatory 
AFDC applicants and recipients - notably, mothers of preschool-age children. The research design 
features that address each of these objectives are discussed briefly below. 

A. The Research Counties 

As noted above, the research design called for nine counties to be selected randomly from the 
25 counties with the largest AFDC caseloads; the selected counties were to be required by the state 
to participate in the study. Tiie random selection of counties enables the evaluation to produce 
findings on program participation patterns and effects that can be generalized to single-parent AFDC 
applicants and recipients who became mandatory for Project Independence during the period under 
study. The nine counties that were selected are listed here (along with the largest city in each) and 
are also shown in Figure 1.3: 

Bay (Panama City) 
Broward (Fort Lauderdale) 
Dade (Miami) 



^^The 25 counties from which the nine research counties were selected represented approximately 90 
percent of the state's AFDC caseload. The process that was used to randomly select counties was designed 
to ensure a large enough research sample for the analysis and to make it highly likely that several of the large 
urban counties, with large AFDC caseloads, would be selected along with some counties with smaller 
caseloads. In most muhi-site evaluations of social programs, the counties, sites, or programs volunteer to be 
included and m.^ht also be selected using other criteria, e.g., the quality of their services or the character of 
their local conditions. 
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FIGURE 1 .3 



THE NINE RESEARCH COUNTIES IN 
THE PROJECT INDEPENDENCE EVALUATION 
(WITH EACH COUNTY'S LARGEST CITY SHOWN IN PARENTHESES) 




Duval (Jacksonville) 
Hillsborough (Tampa) 
Lee (Fort Myers) 
Orange (Orlando) 
Pinellas (St. Petersburg) 
Volusia (Daytona Beach) 

Demographic and economic characteristics of the nine selected counties, and the state as a whole, are 
summarized in Table 1.1. Together these counties included 58 percent of the state's AFDC caseload 
at the start of the evaluation in July 1990. As noted above, they also include Florida's eight largest 
cities - Jacksonville (Duval), Miami (Dade), Tampa (Hillsborough), St. Petersburg (Pinellas), 
Hialeah (Dade), Orlando (Orange), Fort Lauderdale (Broward), and Hollywood (Broward) - as well 
as both suburban and rural areas. The proportion of the research sample represented by each county 
(the second row of Table 1.1) is similar to the proportion of the nine-county AFDC caseload 
represented by each county (the fourth row of Table 1.1). 

As shown in the first two coluimis of Table 1.1, the nine counties as a group are similar to 
Florida as a whole on a number of characteristics - e.g., poverty rates, unemployment rates, and 
percentage employed in the service sector (the most likely source of employment for welfare 
recipients). However, the nine counties have a smaller percentage of their population living in 
rural areas because, as noted above, counties with very small AFDC caseloads are not represented in 
the study. The nine counties also include a larger percentage of Hispanics than does the state overall 
because Dade County includes more than 60 percent of the state's Hispanic population. 

Table 1.1 also indicates that the nine research counties represent a diverse range of 
demographic and economic conditions. Dade County is the largest of the state's 67 counties in terms 
of both population and AFDC caseload, whereas Bay County, the smallest in the study, ranked twenty- 
fourth in population and twenty-second in the size of its AFDC caseload at the start of the study. The 
poverty rates in 1989 ranged from 14 percent in Dade to 6 percent in Lee and Pinellas, and 
unemployment rates in July 1990 ranged from almost 7 percent in Dade to less than 5 percent in Lee 
and Pinellas. The percentage of workers employed in the service sector in 1990 ranged from a high 
of 36 percent in Orange to a low of 26 percent in Bay and Duval. 

Ir^ summary, the information in Table 1.1 suggests that the counties in the evaluation are 
representative of the state in terms of both their average characteristics and their diversity. This 
condition enhances the evaluation's capacity to produce results that can be generalized to the portion 
of the state's AFDC caseload that became mandatory for Project Independence during the period under 
study — or at least to the approximately 90 percent of this AFDC caseload represented by the 25 
counties from which the nine were selected. 

B. Random Assignment of Individuals 

Research on welfare dynamics has shown that there is normally a substantial amount of 



^■^The data presented in Table 1 . 1 ref.sct the available measures of conditions in Florida during the random 
assignment period (July 1990 through August 1991), 
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TABLE 1.1 



SELECTED CHARACTERISTICS OF THE NINE COUNTIES 
IN THE PROJECT INDEPENDENCE EVALUATION 



Characteristic 

Research characteristics 

Research sample size 

Percent of the total 9-county 
research sample 

AFDC characteristics 

Size of the AFDC caseload. 
July 1990 

Percent of the total 
9-county AFDC 
caseload. July 1990 

AFDC caseload growth. 
July 1990 -July 1991 (%) 

Population characteristics 

Population. 1990 

Percent of the 9-county 
l'Of)Ulation, 1990 

Population growth. 1990-91 (%) 

Percent of the population. 1990 
African-American 

Hic,f)anic 

Population living in 
rural areas. 1990 (%) 

f'uverty rate, 1989 (%) 



Total for Total for the 

Florida^ 9 Counties _ Ba^ Browafd_. ^ Da de Duval Hillsboroug h Lee Orange Pi nellas Volu sia 



n/a 
n/a 

138.963 

n/a 

24.0 



n/a 
2.0 

lv'^.1 

12.2 

15.2 
9.0 



18.237 
100.0 



459 
2.5 



2,240 6,021 



12.3 33.0 



2.017 
11.1 



2,415 687 1,749 



13.2 3.8 



9.6 



81.046 1.382 



8,538 30,128 10,430 



10,690 2,269 7.237 



100.0 



24.6 



1.7 



16.6 



10.5 37.2 



42.9 22.6 



12.9 



17.3 



13.2 2.8 



24.6 30.0 



8.9 



28.9 



1,739 
9.5 

7,322 

9.0 
17.1 



100.0 
1.5 

15.0 
18.5 

4.4 

9.6 



1.8 
1.2 

10.7 
1.8 

15.2 
11.2 



i7.a 

1.8 

14.9 
8.6 

1.1 
7.1 



27.4 
1.3 

19.1 
49.2 

1.2 
14.2 



9.5 
1.3 

24.1 
2.6 

1.2 
9.8 



11.8 4.7 
1.1 2.5 



12.8 
12.8 



6.4 
4.5 



10.8 13.8 
9.5 6.1 



9.6 
3.5 

14.8 
9.6 

7.0 
7.8 



12.1 
0.4 

7.6 
2.4 

0.4 
6.2 



910 



5.0 



3,050 



3.8 



25.3 



12.937.926 7,061,576 126,994 1,255,468 1,937,094 672,971 834,054 335.113 677.491 851.659 370,712 



5.2 
1.6 



8.9 
4.0 



7.9 



(continued) 



TABLE 1.1 (continued) 
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Total for Total for the 

Characteristic Fio/ida . 9 Counties Bay Broward Dade Duval Hillsborough Lee Orange Pinellas Volusia 

Labor nriaxKet characteristics 
Unemployment rate, 

July 1990 (%) 6.1 5.5 6.6 5.6 6.8 5.5 5.1 4.6 5.5 4.9 5.2 

Percentage change in 
unemployment rate, 

July 1990 - July 1991 (a) 32.8 32.8 1.5 42.9 35.3 30.1 27.5 56.5 32.7 36.7 40.4 
Percent of all workers employed 

in the service sector, 1990 30.9 31.8 25.6 32.9 32.1 25.7 31.8 28.8 35.9 33.1 31.1 
Growth in service -sector 

employment, 1989 -90 (%}^ 4.6_ _4J 4.e_ _\.7 3.5 6^ 37 67 3.5 23_ 

SOURCES: AFDC characteristics are from statistical data for each county published by Policy Development Economic Services Budget and Analysis, 
provided to MDRC by the state of Florida; data on the population living in rural areas and on ethnicity are from the" 1990 U.S. Census of Population; 
other population characteristics and labor market characteristics are from the 1990, 1991 , 1992, and 1993 Florida County Comparisons. 

NOTES: Where data are not applicable, "n/a" Is used. 

(a) This percentage change was calculated by taking the difference in the unemployment rate between July 1990 and July 1991 and 
dividing it by the July 1990 unemployment rate. 



turnover within the caseload, with many welfare recipients leaving welfare without any program 
assistance and many subsequently returning to the rolls. Thus, without a random assignment 
research design, it is especially difficult to determine the extent to which a program increases (or 
decreases) the rate at which welfare recipients take jobs and stay off the welfare rolls above and 
beyond what they would have done on their own. The Project Independence evaluation used random 
assignment to create two groups of AFDC applicants and recipients for whom Project Independence 
was mandatory: 



A program group, who had access to Project Independence's services and was 
subject to its participation requirements and to possible AFDC grant reductions 
in the event of noncompliance with program rules. 

A control group, who, for a period of two years following random 
assignment, was not required or permitted to participate in Project 
Independence (and, consequently, was not subject to possible AFDC grant 
reductions for noncompliance), and was not allowed to receive its case 
management or employment and training services. However, control group 
members could seek other services on their own. They could also receive 
child care and college tuition assistance under the same guidelines and priorities 
as Project Independence participants.^^ 



Use of a randomly generated control group provides the best information on what would have 
happened to eligible clients in the absence of the program. Since the program and control groups were 
created randomly, the two groups do not differ systematically on both measurable and unmeasurable 
characteristics. Thus, any subsequent differences found between the two groups can be attributed to 
the Project Independence treatment to which program group members were exposed and control group 
members not exposed. The program effects, oi impacts, presented in this report are the differences, 
over the post-random assignment period, between program and control group members' employment, 
earnings, and AFDC receipt. 

In order for this design to yield accurate and reliable estimates of impacts, three important 



^^See Bane and Ellwood, 1983; Pavetti, 1992; and Friedlander and Burtless, 1995. 

^^Exempt applicants and recipients, as well as ongoing AFDC recipients who were participating (or had 
recently participated) in Project Independence at the time of their redetermination hearing, were excluded from 
the random assignment process. In 1990, Project Independence began its program for unemployed parents 
in two-parent households that were receiving AFDC, and these individuals were also excluded from random 
assignment. 

^^Being a member of the evaluation's control group did not affect a person's eligibility for entitlements 
such as AFDC, Medicaid, and Food Stamps. In addition, the status of child care assistance as an 
"entitlement" for welfare recipients engaged in education and training services has been vigorously debated 
in Florida and other states. For purposes of the Project Independence evaluation, it was felt that if members 
of tlie control group decided to pursue employment and training services on their own. they were eligible to 
receive equivalent HRS-funded child care, as well as tuition assistance at community colleges. Control group 
members were also given a list of other. non-Project Independence employment and training services in the 
community. 
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conditions must be met. First, there must be a sufficiently large number of individuals in the sample 
to reduce the likelihood that measured differences are due to chance. As discussed below, the Project 
Independence research sample includes 18,237 individuals - a sample size sufficient to detect policy- 
relevant differences in earnings and AFDC receipt reliably. Second, data collection sources and 
strategies must ensure that equivalent types of outcome information are obtained for both program and 
control group members. As discussed in Chapter 2, MDRC used Florida's administrative records data 
systems to collect earnings, AFDC, and Food Stamp information for both program and control group 
members. 

Finally, control group members must be restricted from receiving the program treatment that 
is being evaluated. Here, the interpretation of the Project Independence evaluation impact estimates 
(and their comparison with findings from other random assignment evaluations of welfare-to-work 
programs) will be complicated by the fact that, contrary to the research design, some control group 
members were exposed to Project Independence services. As discussed in Chapter 3, however, more 
than half of the control group members who were exposed to Project Independence attended only a 
program orientation rjid did not enroll in employment-related activities. Others might have received 
similar services in the community if they had been prevented from participating in Project 
Independence as intended. In addition, in spite of the control group's exposure to Project 
Independence, the program group was still much more likely to participate in employment-related 
activities than were control group members, a measured difference as large as, or larger than, those 
found in past evaluations. As a result, and as concluded in Chapter 5, the control group's exposure 
to Project Independence is likely to have only modestly affected the levels of program impacts, and 
does not appear to have affected the pattern of impacts over time or across subgroups of the research 
sample. 

C. The Early Point of Random Assignment 

Figure 1.4 illustrates the random assignment research design. The random assignment 
procedures were implemented at the HRS Public Assistance Units after AFDC applicants and recipients 
had been determined to be mandatory for Project Independence but before they were actually referred 
to a formal program orientation. This allows the evaluation to capture even very early effects of 
Project Independence, beginning when an individual first learns of the program - on applying for 
AFDC or (in the case of current recipients) when seeking redetermination for ongoing AFDC 
eligibility. For example, if the prospect of having to attend a Project Independence orientation session 
encouraged some people to instead find employment and leave the welfare rolls (or not to pursue their 
application for welfare), that employment and those welfare savings would have been effects of Project 
Independence captured in this evaluation. Other studies of welfare-to-work programs, which carry out 
random assignment at program orientation, may not fully measure such effects. On the other hand, 
this early placement of random assignment also means that the program group includes some 
individuals who did not go on to attend orientation and who may not have been affected by the 
program's employment-related activities and support services. Therefore, the rates of participation in 
post-orientation activities, and the estimated average effects of the program, may be lower than they 
would have been if random assignment had taken place when individuals attended orientation. 

D. The Research Sample and Key Subgroups 

The research sample for the Project Independence evaluation consists of 18,237 AFDC 
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FIGURE 1.4 

OVERVIEW OF RANDOM ASSIGNMENT PROCEDURES 
FOR THE PROJECT INDEPENDENCE EVALUATION 



AFDC Application or 
Redetermination at 
Public Assistance Unit 



Mandatory for Project Independence? 



j Yes 



Project Independence Evaluation 
Explained and 
Background Information Form Completed 



No 



Exit 



Random Assignment at 
Public Assistance Unit 



I Program Group 



Control Group 



Staff Follows Up, 
Reschedules | 
Orientation 



No 



Referred to 
Project Independence 
Orientation 



Attend Project 
Independence Orientation? 



Yes 



Referred to 
Project Independence 
Activities and Services 



May not participate in 

Project Independence for 2 years 

Not subject to participation 
mandate or sanctions for 2 years 

May seek training and employment 
services from other sources 

Eligible for HRS-funded child care 
while in self-initiated activities, on 
same terms as Project Independence 
participants 

Eligible for tuition assistance 
on same terms as Project 
Independence participants 

No effect on entitlements such as 
AFDC, Medicaid, and Food Stamps 
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applicants and recipients for whom Project Independence was mandatory. Of these, 4,724 (26 percent) 
were randomly assigned to the control group and 13,513 (74 percent) were randomly assigned to the 
program group. This sample is large enou^ to permit the analysis to reliably detect overall program 
effects on employment, earnings, and AFDC receipt (assuming that these effects are at least as large 
as those typically found in previous evaluations of welfare-to-work programs). It should be noted that 
the Project Independence research sample has a substantially lower proportion of people who were 
already receiving AFDC at the time of random assignment (and, consequently, a substantially higher 
proportion of applicants) than has been the case in other evaluations of welfare-to-work programs. 
This is because random assignment for this evaluation was initiated approximately three years after 
Project Independence began operating in the research counties. As a result, a large proportion of the 
ongoing AFDC recipients who appeared for their AFDC redetermination appointments during the 
random assignment period were already registered for Project Independence. In order to avoid having 
to discontinue Project Independence services for those who might have been randomly assigned to the 
control group, only ongoing recipients who were newly mandatory for Project Independence (e.g., 
because their youngest child had turned three), and who were not currently registered for the program, 
were randomly assigned. The evaluation's overall findings on program participation and program 
effects are influenced most heavily by the behavior of applicants and reapplicants, who make up 88 
percent of the research sample. Thus, the research sample reflects the flow of newly mandatory 
individuals into a mature, ongoing welfare-to-work program. However, caution should be exercised 
in comparing the results to those of other studies whose research samples include a more even mix of 
applicants and recipients. 

Another key objective of the evaluation was to determine the relative effectiveness of Project 
Independence for different groups of AFDC applicants and recipients, information that would be useful 
in refining program operating and service delivery strategies. An important conclusion of the first 
report on Project Independence was that the earnings impacts were concentrated among those 
individuals with no preschool-age children, those who were classified as job-ready, and those who had 
received AFDC for a total of two years or more (as adults) prior to their entry into the study. The 
report also found that the program was somewhat more effective for individuals who were referred 
to the program early in the evaluation (i.e., the early cohort) than it was for those who were referred 
later (i.e., the late cohort) when caseloads increased and resources did not. 

The current report extends the analysis of participation in employment-related activities and 
program impacts for subgroups defined by: early or late entry into the research sample, age of the 
youngest child (when the parent entered the study), and job-readiness status. Other subgroups are also 
examined, but in less detail. 

IIL The Significance of the Project Independence Evaluation 

In addition to providing the state of Florida with an assessment of the overall effectiveness and 
cost-effectiveness of Project Independence, this evaluation adds to knowledge about JOBS programs 
and their effectiveness under various conditions and implementation strategies. For example, because 
the Project Independence evaluation began three years after Project Independence was initiated and 
more than a year after it became Florida's JOBS program, its findings reflect the results of a mature 
JOBS program. At the same time, the changes that occurred in Florida during the evaluation period, 
although somewhat extreme compared to the nation as a whole, provided a unique opportunity to 
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observe the evolution of a JOBS program from a relatively close approximation of its intended model 
to a program operating under growing stress from increasing caseloads, stagnating resource 
commitments, and a declining labor market. Finally, the findings on program participation and 
program effects for single parents with preschool-age children are among the most extensive available 
for a JOBS program. Previous evaluations have provided only limited findings concerning single 
parents with children between the ages of three and five.^'^ 

The Project Independence evaluation also provides a useful contrast to what has been learned 
from other studies of welfare-to-work programs, including the welfare-to-work programs of the 1980s 
- many of which also stressed labor force attachment services - and the recently completed 
evaluation of California's JOBS program (GAIN), another important random assignment evaluation 
of a JOBS program for which there are comparable data. The Project Independence and GAIN 
evaluations complement each other in many ways, providing important lessons for other states and for 
federal policy related to JOBS and welfare reform more broadly. For example, Project Independence's 
heavy emphasis on independent job search activities for most of its caseload contrasts with GAIN'S 
emphasis on basic education for a large proportion of its program registrants. In 1991, Project 
Independence led the nation's JOBS programs with 50 percent of its participants in job search 
activities, whereas GAIN was one of four programs with more than 50 percent of its JOBS participants 
in basic education. Results from the two evaluations provide information about the consequences 
of choices that states have made about the mix, targeting, and costs of JOBS services. Florida and 
California also differ in their AFDC grant levels, with Florida ranking in the lower part of the range 
and California ranking near the top.^^ Results from the two evaluations will shed light on the 
different types of incentives JOBS participants faced in supplementing and replacing welfare with 
earnings. 



IV, An Overview of This Report 

The next chapter describes the data sources used in this report and examines the background 
characteristics of individuals in the research sample. 

Chapter 3 presents the findings on Project Independence's impact on participation in 
employment- related activities for the full research sample . These findings are the foundation for the 
analysis of the costs of providing employment-related services to program and control group members 
(Chapter 4), and they also provide a context for interpreting the analysis of the program's impacts on 
earnings and welfare receipt for the full sample (Chapter 5). 

Chapter 6 analyzes participation patterns and two-year impacts for selected subgroups of the 
research sample. It addresses several questions left open by the earlier chapters' findings for the full 
research sample: the relative effectiveness of Project Independence for early and later entrants into 



^ See Ffiedlander and Goldman, 1988; and Riccio, Friedlander, and Freedman, 1994. 
^^See Greenberg, 1992. 

^^In January 1991, the basic AFDC grant in Florida for a family of three was $294; 14 states had lower 
grant levels. In that same month, the basic AFDC grant for a family of three in California was S607; only 
Alaska had a higher grant level. 
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the research sample; for single parents with preschool-age children compared to those with older 
children; and for subgroups defined by the Project Independence job-readiness criteria, prior AFDC 
receipt, and other background characteristics. Finally, it presents county-specific findings for the nine 
research counties. 

Chapter 7 analyzes the economic benefits of Project Independence and compares them to the 
cost estimates discussed in Chapter 4. These benefit-cost estimates are presented from several 
perspectives: those of AFDC applicants and recipients in the research sample, government budgets 
(and taxpayers), and society as a whole (which combines the individual and taxpayer perspectives). 
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CHAPTER 2 



DATA SOURCES AND BACKGROUND 
CHARACTERISTICS OF THE RESEARCH SAMPLE 



The first section of this chapter describes the data sources used for this report, the subsamples 
for which the data were collected, and the time periods to which the data apply. The second section 
examines the background characteristics of the foil research ^sample and selected subgroups. 



I. Data Sources 

Table 2.1 lists the general types of information that were collected for individuals in the 
research sample: baseline data (i.e,, background information collected at the time each sample member 
entered the study), state administrative records data, survey data, and Project Independence casefile 
data. Baseline data and infonnatici from state administrative records were collected for all program 
and control group meuibers. The survey and casefile data were collected for randomly selected 
subsamples. Two considerations guided the selection of the survey and casefile subsamples. First, 
a key objective of the analysis was to produce findings that reflected the experiences of the research 
sample as a whole, and thus to increase their applicability to the entire state and the foil random 
assignment period. As a result, the survey and casefile subsamples include individuals from all nine 
research counties who entered the study both early and late in the random assignment period. 
However, it was also important to contain costs associated with long data collection periods (potentially 
as long as 14 months for the two surveys) and with tracking sample members over wide geographic 
areas. Therefore, the survey and casefile subsamples were concentrated in the middle of the random 
assignment period (September 1990 through May 1991)^ and in the more densely populated 
geographic areas in each of the nine research counties. These limitations mean that care should be 
taken in generalizing the survey and casefile findings to the foil sample. However, the sampling frame 
from which these subsamples were drawh includes 56 percent of the foil research sample and reflects 
much of the diversity represented in the nine counties. It also enables the data collected to reflect 
many of the experiences of those who entered the study over the foil random assignment period. 

MDRC also administered a survey to all Project Independence staff in the research counties, 
conducted on-site observations of program activities, and interviewed Project Independence staff to 
learn about how the program was implemented and operated. Finally, state fiscal data were used to 
calculate the costs of providing employment-related services for members of the research sample. The 
remainder of this section of the chapter provides a detailed descriptions of each data source. 

A. Baseline Data 

Data collection for the Project Independence evaluation can be seen as a series of snapshots 
taken of individuals in the research sample over the period of the evaluation. The first of these 



^For the 24--month survey, for example, sample members randomly assigned in September 1990 were 
surveyed in September 1992 and those randomly assigned in May 1991 were surveyed in May 1993. The 
period between September 1992 and May 1993 represents a nine-month survey administialion period, five 
months shorter than if the subsample covered the full random assignment period. 
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TABLE 2.1 



DATA SOURCES AND SAMPLES FOR THE PROJECT INDEPENDENCE EVALUATION 



Data Source 

Baseline data 

Background Information 
Forms (BIFs) 

State admirjistratiye 
records 

Unemployment Insurance 
earnings records 



7* AFDC and Food Stamp 
payment records 



Surveys 

12 -month survey 

of program group members 



24 -month survey 

of program and control 

group members 



Sample for Whom 
Data Were Collected 



All program and 
control group members 



All program and 
control group members 



All program and 
control group members 



Number of Sam pie 
Members for Whom 
Data Were Collected 



Random Assignment Dates 
of Sample Members for 
Whom Data Were Collected 



Follo w-up Period Cover e d by the Data 



Full sample 
Program group 
Control group 



Full sample 
Program group 
Control group 



Full sample 
Program group 
Control group 



Random subsample of progra/n 
group members from the 
Public Assistance Units sen/ing 
1 8 of the largest urban centers 
in the 9 research counties 

Random subsample of program 
and control group members 
from the Public Assistance 
Units sen/ing 10 of the largest 
urban centers in the 9 research 
counties 



18,237 
13,513 
4,724 



18.237 
13,513 
4,724 



July 1990 - August 1991 Data reported as of random assignment date 



July 1990 - August 1991 



5 calendar quarters prior to random assignment 
and 9 calendar quarters including and following 
that of random assignment for the full sample; 

12 calendar quarters including and following that 
of random assignment for an early cohort of the 
research sample 

8 calendar quarters prior to random assignment 
and 10 calendar quarters including and folfowing 
that of random assignment for the full sample; 

13 calendar quarters including and following that 
of random assignment for an early cohort of the 
research sample 



Program group 916 September 1990 - May 1991 12 months following random assignment 



18,237 July 1990 - August 1991 
13,513 
4,724 



Full sample 
Program group 
Control group 



1,029 
520 
509 



September 1990 - May 1991 24 months following random assignment 
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TABLE 2.1 (continued) 



Data Source 

Staff Activities and 
Attitudes Survey 

Project Independence 
casefties (a) 

Casefile data for the 
12 month program 
flow analysis 



Casefile data for the 
24 -month survey 
verification analysis 



Sample for Whom 
Data Were Collected 



Number of Sample 
Members for Whom 
Data Were Collected 



All Project Independence Unit 
supervisoiB and case managers 



Random subsample of program 
group members from the 
Public Assistance Units serving 
1 2 of the largest urban centers 
in ihe 9 research counties 

Random subsample of program 
and control group members 
who responded to the 
24-month survey 



Random Assignment Dates 
of Sample Members for 
Whom Data Were Collected 



Follow-up Period Covered by the Data 



Full sample 
Supervisors 
Case managers 



Full sample 
Program group 
Control group 



190 
26 
164 



Not applicable Survey administered in September and 
October 1991 



Program group 725 September 1990 - May 1991 12 months following random assignment 



400 
150 
250 



September 1990 - May 1991 24 months following random assignment 



NOTES Additional iniormatlon about program operating strategies and organizational issues was obtained through MDRC field research and in~person interviews with 
Project Independence staff in 1991 . Sources of data for the analysis of program costs are noted in the text of Chapter 2 and discussed in detail in Chapter 4. 

(a) Because some sample members did not have contact with Project Independence, program staff did not create casefiles for them. These 
sample sizes include sample members for whom casefiles could not be located as well as those for whom casefile data were obtained. 
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snapshots (referred to as "baseline data") was taken just before random assignment - the point at 
which each sample member entered the study. Since random assignment was used to create the 
program and control groups, the initial pictures of each group should look the same (as indicated later 
in this chapter). The baseline data provide answers to questions about who is being studied and help 
place the Proj'Xt Independence research sample in the context of AFDC applicants and recipients who 
were studied in evaluations of other welfare -to-work programs. The baseline data are also used to 
address questions about who benefits most (or least) from Project Independence. 

The primary source of baseline data is the Background Information Form (BIF) developed by 
MDRC in collaboration with HRS Public Assistance Specialists (PASs). It was designed to collect 
demographic and identifying information for all Project Independence-mandatory AFDC applicants and 
recipients at the time they entered the research sample. Each BIF was completed by the PAS in 
consultation with the sample member, and the data reflect the characteristics of each sample member 
as of the date she was randomly assigned. (People were randomly assigned throughout the period from 
July 1990 through August 1991.) These forms were mailed to MDRC after random assignment was 
completed. Partial BIF data are available for all program and control group members, while more 
complete BIF data are available for 97 percent of each group. ^ BIF data are used later in this chapter 
to describe the research sample members. They are also used in the impact analysis to increase the 
precision of the estimates reported in Chapter 5 and to identify the subgroups that are the focus of 
Chapter 6. 

B. State Administrative Records 

Because people are randomly assigned to the program and control groups, there should be no 
systematic differences between the two groups at the point of random assignment. Therefore, 
assuming that the research design has been implemented correctly, subsequent differences between the 
two groups that appear in later "snapshots" (referred to as "outcome data") can be attributed to Project 
Independence. The primary source of outcome data for this report was administrative records kept 
by the state of Florida. These include information on sample members' earnings reported by 
employers to the Unemployment Insurance (UI) system. The state records also include AFDC and 
Food Stamp payments made to sample members during the evaluation period. The rules for recording 
information in these records apply equally to all state residents. As a result, tliey provide accurate and 
unbiased measures of employment, earnings, and welfare receipt for both the program and control 
groups. Following is a more detailed description of the state administrative records data used in this 
report. 



^Some information from the BIF was collected by phone at the time of random assignment. When a 
Public Assistance Specialist called MDRC to request a random assignment, she or he v/as asked to report 
seven items: the name of the county and the location of the Public Assistance Unit, whether the individual 
was an AFDC applicant or an ongoing recipient, and the individual's name, social security number, birth date, 
and AFDC case number (if she had already been given one). MDRC random assigmnent clerks then 
keypunched these items into the random assignment data base, known as the Random Assignment Record 
(RAR). Since all the RAR items were required before random assigmnent could be completed, these data are 
available for all sample members. However, fully completed BIFs could not be obtained from the Public 
Assistance Units for all sample members. Some BIFs may have been lost at the Public Assistance Units. It 
is also possible that BIFs were not completed for some sample members and that Public Assistance Specialists 
used some other source of identifying information to complete random assignment. 
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1. Unemployment Insurance earnings records ^ Earnings reported by employers to the 
state's UI system provide measures of employment, earnings, and receipt of UI benefits by calendar 
quarter (January through March, April through June, July through September, and October through 
December) for all 18,237 program and control group members. Data were collected for the period 
April 1989 (five quarters prior to the first random assignment) through September 1993. For some 
sample members, the calendar quarter in which random assignment took p!ace may have included some 
earnings secured before the actual day of random assignment; therefore, for all sample members, the 
quarter in which random assignment occurred (quarter 1 of the follow-up period) is not counted in the 
analysis of program effects. Eight full quarters of post-random assignment follow-up data (quarters 
2 through 9) are available for the full sample, and 11 full quarters of follow-up data (quarters 2 
through 12) are available for sample members randomly assigned from July through December 1990. 
In addition, the UI data include a minimum of five calendar quarters of earnings data prior to each 
sample member's date of random assignment. These data are used to increase the precision of the 
impact estimates by controlling for small random differences in the background characteristics of the 
program and control groups. 

Florida's UI system is statewide and therefore provides data on earnings and UI benefits that 
sample members obtained in both research and non-research counties within the state. However, these 
data are not available for earnings obtained in another state or for jobs not usually covered by the UI 
system (e.g., some domestic work or informal child care). Such earnings will not be measured in this 
study and will not appear in the data. 

2. AFDC and Food Stamp payment records . Monthly AFDC payments and Food 
Stamp issuance records were obtained for all 18,237 program and control group members from the 
state's computerized Assistance Payments System (APS) and FLORIDA System.^ These data cover 
the period from June 1988 (two years prior to the first random assignment) through September 1993. 
For the analysis in this report, AFDC payments and Food Stamp issuance amounts were aggregated 
into calendar quarters to match the intervals covered by the UI earnings data. A total of nine full 
quarters of AFDC follow-up data (not including the quarter of random assignment) are available for 
the full research sample, and 12 quarters of follow-up data (not including the quarter of random 
assignment) are available for sample members who were randomly assigned from July through 
December 1990. In addition, the data include a minimum of eight calendar quarters of AFDC and 
Food Stamp data prior to each sample member's date of random assignment. As in the case of the 
pre-random assignment UI data, these data are used to increase the impact estimates' precision by 
controlling for small random differences in the two research groups' background characteristics. 

Florida's welfare payments system is also statewide and therefore provides data on AFDC 
payments and Food Stamp issuances that sample members received in both research and non-research 
counties within the state. However, these data are not available for welfare payments made to 



^Beginning in June 1991, HRS began converting its automated public assistance benefits and service 
system from APS to the Florida On-1 jne Recipient Integrated Data Access System (FLORIDA). This required 
that extra care be taken to ensure that data were collected accurately for all sample members as their AFDC 
cases were converted from APS to FLORIDA. 

"^The primary exception to this would be a sample member's moving to a non- research county and 
receiving AFDC under a new case number assigned there. Those payments will not be measured and will 

(continued...) 
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sample members by another state. These will not be measured in this study and will not appear in the 
data. 

C. Surveys of Research Sample Members 

Administrative records provide a useful but limited picture of the experiences program and 
control group members had after entering the study. For example, while they provide a complete 
picture of sample members' receipt of AFDC and Food Stamp benefits, they do not provide 
information about their use of services designed to help them become less dependent on these benefits; 
nor do they provide insights into their attitudes toward receiving welfare. Also, the UI data provide 
aggregate information about quarterly earnings, but nothing about the characteristics of the jobs held. 
For these types of information, the evaluation drew on data provided by the sample members 
themselves through two surveys: one administered 12 months after random assignment and the other 
administered 24 months after random assignment. These surveys are described below. 

1. The 12*month survey of program group members ^ MDRC fielded a 30-minute 
survey for a subsample of program group members randomly selected from among a'.l those who were 
randomly assigned to the program group between September 1990 and May 1991 in the Public 
Assistance Units serving the largest urban centers in the nine research counties.^ The 12-month 
survey subsample analyzed in this report consists of the 916 individuals in this group who completed 
the survey interview, which was conducted by telephone or in-person.^ 

Fielded between January and May 1992, the 12-month survey asked questions about program 
group members' participation in Project Independence activities, attitudes toward the program, and use 
of child care services during the 12 months following the date on which they had been randomly 
assigned. ITie data are used in Chapter 3 (to provide information about whether Project Independence 
participants found the program's services helpful) and Chapter 4 (to measure child care use for the cost 
analysis). 

2* The 24-month survey of p ro gram and control group members ^ MDRC also fielded 
a 30-minute survey for randomly selected subsamples of program and control group members. 
Individuals in these subsamples entered the research sample between September 1990 and May 1991. 
Like the 12-month survey subsample, the 24-month survey subsample included only individuals who 



continued) 

not appear in the data. However, most cross-county migrants are assigned their original case number if they 
reapply for AFDC in another county. If sample members received AFDC under more tiian one case number 
- for example, because of the conversion from APS to FLORIDA or because they were given a new number 
in one of the research counties for other reasons - these payments are included in the analysis. 

^The 12-month survey subsample was concentrated in 18 large urban areas within the counties: three from 
each of the three largest counties (Dade, Duval, and Hillsborough); two from the three middle-sized counties 
(Broward, Orange, and Pinellas); and one from each of the three smallest counties (Bay, Lee, and Volusia). 

^In My 1,134 program group members were randomly selected for the follow-up survey. The 916 
respondents in the 12-month survey subsample represent a response rate of 81 percent. An analysis of survey 
nonresponse bias indicated that there were virtually no systematic differences in measured characteristics 
between the subsample who completed the survey interview and the 1,134 originally selected. 
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were randomly assigned from the Public Assistance Units located in large urban centers within the nine 
research counties 7 The survey subsample analyzed in this report consists of the 520 program group 
members and 509 control group members who completed the survey interview, which was conducted 
by telephone or in-person.^ 

The survey was fielded between September 1992 and May 1993, 24 months after each sample 
member's random assignment month. The questions asked about their participation in employment- 
related activities, the characteristics of the jobs they held, and the general quality of their lives during 
the two years following random assignment. Data from the 24~month survey are used in Chapter 3 
to analyze patterns of participation in employment-related activities for program and control group 
members during the two years following random assignment. These findings, along with findings from 
Project Independence casefile data, were used in Chapter 4 as the basis for estimating costs of 
providing services for the program and control group members,^ In Chapter 5, the 24-month survey 
data are used to analyze Project Independence's impact on job characteristics and noneconomic 
outcomes. 

D. Project Independence Casefiles 

The survey data provided a general picture of sample members* participation in employment- 
related activities, but they proved less useful as sources of specific information about the types and 
length of interactioFx they had with Project Independence. This is due, in part, to the fact that many 
aspects of Project Independence overlap with other agencies in the community. For example, sample 
members who were referred to a community college education or training program by Project 
Independence may not have remembered the referral, only their connection to the college itself. In 
addition. Project Independence staff often use specific rules and procedures to classify participanvj' 
status in the program, and individual participants frequently cannot differentiate among these statuses. 
For example, participants who are not actually engaged in activities at a particular time may be in the 
process of being referred to an activity, temporarily deferred from participation, or prohibited from 
participating because of a change in their AFDC status. Casefile data are generally more accurate for 
capturing these important differences. Finally, some survey respondents had difficulty recalling the 
dates on which they had participated in various aspects of Project Independence during the follow-up 
periods. 



^The 24-month survey subsample was concentrated in 10 large urban areas within the counties: two in 
Dade and one from each of the other eight counties. 

^In all, 630 program group members and 630 control group members were randomly selected for this 
follow-up survey. The 520 and 509 members of the 24- month survey subsamples represent response rates 
of 83 and 81 percent for the program and control groups, respectively. An analysis of survey nonresponse 
bias (Appendix B) indicated that there were virtually no systematic differences in measured characteristics 
between the subsample who completed the survey interview and the 1,260 originally selected. In addition, 
there were no systematic differences between program and control group survey respondents. 

^As discussed later in this chapter and in Chapter 3, 24-month survty respondents, particularly control 
group members, were limited in their capacity to identify activities that occurred through Project 
Independence. As a result, the 24-month survey findings for the cost analysis were supplemented with 
findings from Project Independence casefile data. 
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To obtain more specific information about sample members' involvement with Project 
Independence, MDRC worked with Project Independence staff to collect data directly from the 
individual casefiles that they maintained. These data were also used as a benchmark for checking the 
accuracy of the 24-month sur\'ey data. The casefile data, and the subsamples for which they were 
obtained, are described below. 

1. Casefile data for the 12-inoDth program flow analysis^ MDRC's first report on 
Project Independence and Chapter 3 of this report present findings on the program group's involvement 
with Project Independence during the first 12 months following their random assignment. The findings 
are based on data collected for a random subsample of 725 program group members (referred to as 
the " 12-month program flow subsample") who were originally selected for the 12-month survey. Like 
the 12-month survey subsample, the 12-month program flow subsample was concentrated in the largest 
urban areas in each of the nine research counties. For 639 members (88 percent) of this 
subsample, the Project Independence casefiles were obtained and data were collected from them.^^ 
These casefiles contain detailed information about program group members' involvement with the 
program - including referrals to program activities, the incidence of participation in activities, length 
of stay in activities, and involvement with formal enforcement procedures. 

Casefiles could not be located for the other 86 members (12 percent) of this subsample, most 
of whom probably never attended a program orientation and so never had a casefile created for them. 
For this group, the 12-month survey data were used, but only for information as to whether they had 
or had not participated in Project Independence; there were insufficient data concerning their 
sanctioning rates and length of participation in the program. 

2. Casefile data for the 24-month survey verification analysis ^ Preliminary analysis 
of the 24-month survey data indicated that a substantial number of control group members reported 
that they had been involved in Project Independence. It was critical to verify the accuracy of this 
information because the research design explicitly specified that control group members were not 
permitted to participate in Project Independence for a period of two years. To the extent that control 
group members received the same Project Independence services as program group members, the 
measured post-random assignment differences between the two groups (i.e., the program impacts) 
would be narrowed. Therefore, MDRC worked with staff in the local Project Independence units to 
locate Project Independence casefiles for a random subsample of 250 control group members and (for 
comparison) 150 program group members who responded to the 24-month survey. These random 



^^^While the 12-month survey was drawn for 18 such areas within the counties, the 12-month program flow 
subsample was confined to 12 of these areas: two from each of the three largest counties (Dade, Duval, and 
Hillsborough) and one from each of the other six counties. 

^^In all, 756 program group members from the 12-month survey subsample were targeted for Project 
Independence casefile data collection. MDRC staff were able to obtain Project Independence casefiles for 639 
(85 percent) of them. An analysis of casefile nonresponse oias indicated that there were virtually no 
systematic differences in measured characteristics between the subsample for whom Project Independence 
casefiles were obtained and the 756 sample members originally selected. 

^"^Casefile data, rather than computerized program tracking records, were required because an assessment 
of data from the relatively new automated Project Independence Information System indicated that it did not 
contain complete information about program group members' participation in program activities. 
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subsainples were drawn to include both individuals who reported participating in Project Independence 
and individuals who did not. 

In all, casefiles were found for 42 percent of the control group members and 77 percent of the 
program group members. Project Independence staff reported that most of the remaining sample 
members probably never made contact with the program and, that, consequently, a Project 
Independence casefile would not have been created for them. However, this casefile data collection 
effort was undertaken in August 1994 — three to four years following the random assignment dates 
of those in the subsample. Project Independence staff reported that if involvement with Project 
Independence had ceased two or more years prior to the data collection period, the casefiles might 
have been shipped off site for archiving or destroyed. As a result, the findings from this effort had 
to be adjusted to account for missing casefiles. These adjustments were based on casefile data used 
in the 12~month program flow analysis. 

For the Project Independence casefiles that were located, MDRC asked Project Independence 
staff to provide information about whether individuals had been served by the program within the two 
years after their random assignment date. Staff were also asked to indicate whether the individuals 
had ever attended job search or job club activities or education or training activities. As discussed in 
Appendix C, these data, along with the casefile data collected earlier in the evaluation, indicate that 
the (self-reported) information on Project Independence participation from the 24-month survey is 
reasonably accurate for the program group. However, according to the Project Independence casefile 
data, control group members appear to have over-reported their involvement with the program. These 
casefile data are used in Chapter 3, which presents the findings on the control group's exposure to 
Project Independence. Chapter 5 also uses this information in discussing a strateg}' for adjusting the 
observed impact estimates to account for the control group's exposure to Project Independence. As 
noted in Chapter 1, this analysis indicates that the control group's exposure to Project Independence 
may have affected the overall level of the impact estimates, though probably only modestly, but is not 
likely to have affected the trends over time or across subgroups. 

E, The Staff Activities and Attitudes Survey 

To describe the ways staff implemented the Project Independence program model in the 
research counties, this report also relies on information from a Staff Activities and Attitudes Survey 
developed by MDRC. It was administered in September and October 1991 to the 26 unit supervisors 
and 164 case managers in the 26 Project Independence units in the nine research counties. 

The survey included a broad range of questions about staff practices regarding several key 
dimensions of Project Independence. It also asked staff about their attitudes toward their job. Project 
Independence, and AFDC recipients; their perceptions of Project Independence's services and goals; 
and their own backgrounds. Because the survey was administered at a single point in time, it does not 
capture the range of attitudes and activities over the full evaluation period. Data from this survey are 
used in Chapter 3 to describe the staffs assessment of the availability and quality of Project 
Independence services. 

F. Staff Interviews and Field Research 

In early 1991, MDRC staff held in-person interviews with Project Independence district 
administrators, unit supervisors, and case managers in each of the nine research counties. During this 
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period, MDRC staff also observed a variety of Project Independence activities. This qualitative 
research was used to learn about program operating strategies and organizational issues. 

G. Cost Data 

To estimate the costs and benefits of Project Independence, the study uses fiscal and program 
participation data provided primarily by HRS. HRS expenditure data were obtained from the quarterly 
Project Independence expenditure reports of HRS's Economic Services Division (especially form ACF- 
332, prepared for the federal government) and covered the period from July 1990 (the start of random 
assignment) through June 1992. MDRC also collected cost data related to services provided by LES, 
the Florida Department of Education, local Central Child Care agencies, and several other sources. 
Chapter 4 presents a detailed description of these data and how they were used for the cost analysis. 



IL Background Characteristics of the Full Research Sample and Key Subgroups 

The full research sample for the Project Independence evaluation consists of 18,237 AFDC 
applicants and recipients from the nine research counties. This group encompasses all of the AFDC 
applicants who were determined to be mandatory for Project Independence during the period from July 
1990 through August 1991. (More than half of these applicants had received AFDC previously under 
their own or their spouse's case.) The sample also includes most of the ongoing AFDC recipients who 
were newly determined to be mandatory during this period. The primary exceptions were ongoing 
recipients who had participated in Project Independence as volunteers within 45 days of their 
redetermination for ongoing AFDC eligibility. These individuals were not included in the research 
sample. 

A total of 13,513 individuals (74 percent of the research sample) were randomly assigned to 
the program group and, therefore, were referred to Project Independence orientation and were subject 
to the program's participation requirements. A total of 4,724 individuals (26 percent of the research 
sample) were assigned to the control group and told that they were not required or permitted to 
participate in Project Independence activities. 

Appendix Table A.l presents the demographic characteristics of the AFDC applicants in the 
program and control groups; Appendix Table A. 2 presents this information for the ongoing AFDC 
recipients. These tables indicate that the random assignment process produced two equivalent 



^^These background characteristics are presented in separate tables because the applicants and recipients 
were randomly assigned using different program-to-control-group ratios. Applicants were assigned at a ratio 
of three program group members for each control group member. In the early part of the random assignment 
period — from July through December 1990 — recipients were also assigned using a 3:1 ratio, but thereafter 
(from January through August 1991), their ratio was lowered to 2:1 to increase the number of recipients in 
the control group and, thus, the statistical reliability of the impact estimates for recipients. The use of separate 
tables for applicants and recipients in Appendix A provides the opportunity to determine whether there were 
differences in measurable characteristics between the program and control groups other than the change in the 
random assignment ratio for recipients. 
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research groups (as was its purpose), with only a few statistically significant^'* demographic 
differences between the program and control groups. 

The "full sample" column of Table 2.2 presents the percentage distribution of selected 
demographic characteristics for the full research sample. These characteristics, are used to define 
subgroups that are the focus of Chapter 6 and are likely to have contributed to differences in program 
participation patterns as well as program impacts and costs. In particular, the chapter focuses on four 
sets of subgroups defined by: (1) the period during which individuals were randomly assigned and 
entered the research sample; (2) age of the youngest child; (3) Project Independence job-readiness 
status; and (4) history of prior AFDC receipt. These subgroups were likely to vary in the degree and 
nature of their involvement with Project Independence and in their likelihood of being able to secure 
employment and leave AFDC without the aid of Project Independence. The characteristics of these 
subgroups are discussed below and should be considered when comparing subgroup participation and 
impact results presented in Chapter 6. 

A. Characteristics of Subgroups Defined bv the Period During Which They Entered the 
Research Sample 

As discussed in Chapter 1 , the Project Independence evaluation occurred during a period when 
Project Independence and its state and local environments were undergoing several important changes. 
As a result, this report analyzes the effectiveness of Project Independence at different points during 
this period. In particular, it compares the experiences of individuals who were randomly assigned and 
entered the study early with those who entered later. The first subgroup (representing 41 percent of 
the full sample) was randomly assigned between July and December 1990. This "early cohort" and 
its experiences reflect the effects of Project Independence when it was operating under relatively 
favorable conditions. The "late cohort" (randomly assigned between January and August 1991, and 
representing 59 percent of the full sample) experienced Project Independence when it was operating 
under the stress of growing caseloads, fixed resources, and increasing unemployment. 

Comparing the experiences of these two subgroups must be informed by an examination of the 
extent to which they were different to begin with because such differences may have contributed to 
differences in the two subgroups' patterns of participation in employment-related activities and in 
program impacts (above and beyond differences associated with changes in the program and its 
environment). The last two columns of Table 2.2 display the distribution of selected demographic 
characteristics for the two cohort subgroups. They show that there were some differences between 
the two subgroups, although most were relatively small (but statistically significant). Notably, the late 
cohort included a slightly higher percentage of individuals who had earnings during the year prior to 
random assignment. This may have been due, in part, to an increase in the number of people losing 
their jobs, as reflected in the state's rising unemployment rate during this period, and turning to AFDC 



^'^When calculated differences are statistically significant (indicated by asterisks in the tables), one can have 
a high level of confidence that the differences are not due to statistical chance. 

^^The information in Table 2.2 was obtained from tlie Background Information Forms (BIFs). As 
discussed earlier, BIFs could not be obtained for all sample members, and some items from the BIFs that were 
obtained are missing for some sample members. Table 2.2 presents percentages of the sample who are 
missing data on selected characteristics. 
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TABLE 2.2 



SELECTED CHARACTERISTICS OF THE FULL SAMPLE AND 
OF SUBGROUPS DEFINED BY RANDOM ASSIGNMENT COHORT. 
AT THE TIME OF RANDOM ASSIGNMENT 



Characteristic and Subgroup 



Subgroups, by 
Random Assignment Cohort 
Sample Full Early Cohort Late Cohort 

Size Sample (7/90 - 12/90) (1/91 - 8/91) 



Ethnicity (%) 
White, non -Hispanic 
Black, non-Hispanic 
Hispanic 
Other 

Data not available 

Primary language (%) 
English 
Spanish 
Other 

Data not available 
Average age (years) 

Number of children (%) 

1 child 

2 children 

3 or more children 
Data not available 

Age of youngest child (%) 
Age 3 to 5 
Age 6 or older 
Data not available 

Total prior AFDC receipt (a) (%) 
First-time applicant 

Applicant with less than 2 years of AFDC receipt 
Recipient with less than 2 years of AFDC receipt 
Applicant with 2 years or more of AFDC receipt 
Recipient with 2 years or more of AFDC receipt 
Data not available 

Education (%) 
High school diploma or GED 
No high school diploma or GED 
Data not available 

Any earnings during the prior year (%) 

Job- readiness status (b) (%) 
Job-ready 
Not job- ready 
Data not available 

Research sample status (%) 
Program group 
Control group 

Random assignment cohort (%) 
Early cohort (July-December 1990) 
Late cohort (January-August 1991) 

Sampl e size 



6.118 
6,878 
4,044 
207 
990 



13,887 
3,232 
303 
815 

18,237 



7,860 
5,575 
3,797 
1,005 



7,211 
9,316 
1,710 



7,120 
4,373 
790 
3,593 
1,029 
1,332 



9.437 
7,543 
1,257 

18,237 



14,936 
1.816 
1.485 



13,513 
4.724 



7.511 
10.726 



33.5 
37.7 
22.2 
1.1 
5.4 



76.1 
17.7 
1.7 
4.5 

32.1 



43.1 
30.6 
20.8 
5.5 



39.5 
51.1 
9.4 



39.0 
24.0 
4.3 
19.7 
5.6 
7.3 



51.7 
41.4 
6.9 

61.1 



81.9 
10.0 
8.1 



74.1 
25.9 



41.2 
58.8 



31.1 
38.3 
25.5 
1.1 
4.0 



74.5 
20.6 
2.2 
2.7 

32.0 



43.0 
30.9 
22.5 
3.6 



41.9 
50.8 
7.4 



39.2 
24.9 
4.0 
21.6 
4.7 
5.6 



51.6 
43.1 
5.3 

59.3 



82.1 
10.9 
6.9 



74.7 
25.3 



100.0 
. 0.0 



35.3 *** 
37.3 
19.9 *** 
1.2 

6.4 *** 



77.3 *** 
15.7 *** 
1 .3 *** 
5.7 *** 



*** 



32.1 



43.2 
30.3 
19.7 
6.9 

37.9 
51.3 
10.8 



39.0 

23.3 ** 
4.6 * 

18.4 *** 
6.3 *** 
8.5 *** 



51.8 
40.1 *** 
8.1 *** 

62.4 *** 



81.8 
9 3 *** 
9.0 *** 



73.6 
26.4 



0.0 
100.0 



18.237 



7.511 



10.726 



(continued) 
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TABLE 2.2 (continued) 



SOURCES: MDRC calculations from Background Information Forms and Florida Unemployment Insurance (Ul) records. 

NOTES: Distributions may not add to 100.0 percent because of rounding. 

A chl-square test or two -tailed t-test was applied to differences between subgroups defined by 
random assignment cohort. 

Statistical significance levels are indicated as *** = 1 percent; ** = 5 percent; * = 10 percent. 

(a) This refers to the total number of months accumulated from one or more spells on an individual's own 
or spouse's AFDC case. It does not include AFDC receipt under a parent's name. 

(b) Sample members are defined as "job-ready" If they had completed at least 1 oth grade or were employed 
for at least 1 2 of the 36 months prior to random assignment. They are defined as "not job-ready" if they did not meet both 
criteria. These definitions are based on those used by Project Independence during the the random assignment period. 
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for income support. The late cohort also included a slightly higher percentage of ongoing AFDC 
recipients and a soniewhat lower percentage of families with children between the ages of three and 
five. 

In general, however, the findings presented in Table 2.2 suggest that, although AFDC (and 
Project Independence) caseloads grew rapidly during 1991, the characteristics of mandatory AFDC 
applicants and recipierits who were referred to Project Independence were not very different from the 
characteristics of those who were referred earlier. Thus, it is more likely that differences in outcomes 
and impacts between the two cohort subgroups were due to changes in Project Independence and its 
environment than to differences in their background characteristics. 

B. Characteristics of Subgroups Defined by the Age of the Youngest Child 

This report also provides participation and impact findings for AFDC applicants and recipients 
with children between the ages of three and five - a group not usually mandated to participate in 
welfare-to-work programs prior to JOBS - as well as for the traditional mandatory group of single 
parents whose youngest child is age six or older. Participation rates and impacts may not be the same 
for the two groups because mothers of preschool-age children are very likely to need child care if they 
are to participate in program activities or become employed. Table 2.2 shows that 40 percent of the 
sample members reported having children between the ages of three and five, while just over half (51 
percent) had a youngest child age six or older. In addition, 43 percent of the sample had only one 
child, and 21 percent had three or more children. 

Appendix Table A. 3 presents the distribution of background characteristics for the two age-of- 
youngest-child subgroups and reveals several important differences.^^ Sample members with 
preschool-age children (i.e., between the ages of three and five) were an average of six and a half 
years younger than those whose youngest child was six or older, but were more likely to have three 
or more children (28 percent compared to 18 percent of those with children age six or older). Also, 
those with preschool-age children were more likely to be ongoing AFDC recipients (rather than 
applicants) at the time of random assignment. There were also snaaller differences between the two 
groups in terms of their ethnicity, primary language, educational attainment, and job-readiness status. 

C. Characteristics of Subgroups Defined by Project Independence Job-Readiness Status 

The Project Independence job-readiness criteria are based on an assessment of the educational 
background and employment history of its participants and are used to determine initial service 
referrals. Table 2.2 shows that 82 percent of the research sample would have been defined as "job- 
ready" under the Project Independence criteria used at the time of their random assignment - i.e., 
they had completed at least the tenth grade or had been employed in at least 12 of the previous 36 



^^For a total of K710 sample members (9 percent of the full research sample), information about the age 
of the youngest child is missing either because the information was not provided on the BIF or because the 
BIF itself was missing. 

^^The percentages reported for subgroups defined by the age of the youngest child in Table A. 3 reflect 
the percentages of those for whom this subgroup information was available. 



-33- 



months. ^ In addition, 61 percent of the sample had some earnings in the year prior lo random 
assignment, and 52 percent had received a high school diploma or GED. 

Appendix Table A. 4 presents selected background characteristics of the two job-readiness 
subgroups as defined by criteria in effect during the random assignment period. It shows, for 
example, that members of the not job-ready subgroup were much more likely to be Hispanic (42 
percent) and to speak Spanish as their primary language (37 percent) than were those defined as job- 
ready (20 percent of whom were Hispanic and 16 percent of whom spoke Spanish as their primar>' 
language). This suggests that limited Enrlish proficiency may have constituted another important 
barrier to employment among many not job-ready subgroup members. Table A. 4 also shows that 
members of the not job-ready subgroup were also more likely to have preschool-age children and to 
have three or more children, suggesting that they may have had more prominent child care needs than 
did the job-ready subgroup. Finally, those in the not job-ready subgroup were more likely to be 
ongoing AFDC recipients and to have received AFDC for two years or more than were those defined 
as job-ready. 

As noted in Chapter 1, Project Independence modified its job-readiness criteria in October 
1991. Specifically, those who had no high school diploma or GED or had worked in fewer than 12 
of the previous 24 months would now be defined as not job-ready. Because this occurred 
approximately three months after the random assignment period ended, it is not likely to have affected 
the initial service referrals of program group members who attended a Project Independence 
orientation. However, an important question for the evaluation is whether the original program model 
was effective for the group that, under the modified job-readiness criteria, would have had the option 
of participating initially in education and training activities. This group consists mainly of members 
of the job-ready subgroup of the research sample who did not have high school diplomas. 
Appendix Table A. 4 indicates that 35 percent of the job-ready subgroup did not have a high school 
diploma and, therefore, would have been defined as not job-ready under the modified criteria. 

D. Characteristics of Subgroups Defined by Prior AFDC Receipt 

Past research indicates that the effects of welfare-to-work programs are related to an 
individual's prior history of AFDC receipt. Also, as noted in Chapter 1, the research sample for 
the Project Independence evaluation is distinctive from those in other MDRC evaluations of welfare-to- 
work programs because of the under-representation of ongoing AFDC recipients and the predominance 
of AFDC applicants. This is important because those just coming on to the AFDC rolls - for the first 
time or after a hiatus from a previous spell on welfare — differ in many ways related to their readiness 
for employment. 



^^For a total of 1,485 sample members {8 percent of the full research sample), BIF data about the highest 
grade completed in school and employment experience were not available, so their job-readiness status could 
not be ascertained. 

'^e percentages presented for these subgroups in Appendix Table A.4 reflect the percentages of sample 
members for whom job-readiness information was available. 

'^^It is not possible to accurately identify sample members' job-readiness status under the revised criteria 
because the BIF asked only for information about employment in the previous 36 months (not the previous 
24 montlis). 

*^^See, e.g., Friedlander, 1988; and Hamilton and Friedlander, 1989. 
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Table 2.2 indicates that 39 percent of the research sample were applying for AFDC for the first 
time. An additional 24 percent of the sample were not receiving AFDC at the time of random 
assignment (i.e., they, too, were applicants) but had received it for less than two years prior to random 
assignment, and 20 percent were not receiving AFDC but had received it for two years or more prior 
to random assignment. Table 2.2 indicates that only 10 percent of the research sample were 
receiving AFDC at the time of random assignment (4 percent had received it for a total of less than 
two years and 6 percent had received it for two years or more).^^ As noted above, this is because 
individuals who had been in Project Independence recently were not included in the sample. During 
the random assignment period, MDRC staff found that most ongoing recipients who were being 
redetermined for AFDC eligibility v;ere already registered for Project Independence. 

Appendix Table A. 5 displays selected demographic characteristics of three subgroups defined 
by how long sample members had received AFDC prior to random assignment:^"^ those who had 
never previously been on their own or their spouse's AFDC case (referred to in this report as "first- 
time applicants"); those who had previously been on their own or their spouse's AFDC case for a total 
of less than two years prior to random assignment (i.e., "short-term" applicants and recipients); and 
those who had previously been on their own or their spouse's AFDC case for a total of two years or 
more prior to random assignment (i.e., "long-term" applicants and recipients). 

Table A. 5 indicates that these subgroups differed on several background characteristics. For 
example, the first-time applicants were more likely to be Hispanic and to speak Spanish as their 
primary language than was either of the other two subgroups. As noted above, limited proficiency in 
English may have constituted an important barrier to employment. However, first-time applicants were 
the least likely to have three or more children and the least likely to have preschool-age children. 
They were also the most likely to be job-ready and to have a high school diploma or GED at the time 
of random assignment. This suggests that first-time applicants faced the fewest barriers to employment 
and, therefore, were the most likely to have been able to get a job and leave AFDC without the help 
of Project Independence. Finally, long-term applicants and recipients were the most likely to have 
three or more children, the least likely to be job-ready, and the least likely to have a high school 
diploma or GED. Each of these factors may have constituted an important barrier to employment. 



^^Information on prior AFDC receipt was reported on the BIF in terms of the total number of months the 
individual had received AFDC under her own or her spouse's name, regardless of the number or duration of 
spells of AFDC receipt this represented. Thus, prior receipt of AFDC was not necessarily continuous and 
was not necessarily recent. Also, prior receipt does not include AFDC received under her parent's name. 

^"^For a total of U332 sample members (7 percent of the full research sample), information about prior 
AFDC receipt is missing. 

^"^The percentages reported for these subgroups in Table A. 5 reflect the percentages of sample members 
for whom BIF data about prior AFDC receipt was available. 

^The subgroup of long-term applicants and recipients used in this report may differ from those used in 
other MDRC reports. For example, the long-term group has often included only those who were receiving 
AFDC at the time of random assignment and had received AFDC for two or more years. See Friedlander, 
1988; and Hamilton and Friedlander, 1989. 
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CHAPTERS 



THE IMPACT OF PROJECT INDEPENDENCE ON 
PARTICIPATION IN EMPLOYMENT-RELATED ACTIVITIES 



Project Independence aims to increase the employability and self-sufficiency of welfare 
recipients through a combination of participation requirements, case management, and employment- 
related activities provided directly by the program (i.e., independent job search and job club) or 
through the program via linkages to education and training institutions, employment services, and 
employers (as sources of both jobs and on-the-job training). To facilitate panicipation, support 
services such as child care and transportation were also built into the program model. The Project 
Independence evaluation was designed to test whether individuals who were exposed to these 
requirements and components were more likely to find work and leave welfare than individuals who 
were not required to participate in these employment-related activities but could pursue them in the 
community on their own initiative. This chapter describes the Project Independence panicipation 
requirements and services to which the program group was exposed and examines the program's 
impact on engaging individuals in employment-related activities. These findings provide a critical 
context for interpreting the analysis of Project Independence's impacts on earnings and welfare 
payments (Chapter 5 and 6) and the analysis of whether the program's costs were outweighed by its 
benefits to individuals and to the government budget (Chapter 7). 

The first section of the chapter presents an analysis of the Project Independence "treatment" 
and the extent to which program group members were exposed to it. MDRC's first report on Project 
Independence described this treatment in detail and analyzed the program group's "flow" through the 
Project Independence program model during the first year following their referral to the program and 
entry into the research sample.^ Section I of this chapter summarizes the key findings from the earlier 
report. 

Section II extends this analysis by presenting findings from the 24-month survey on the 
program group's participation in employment-related activities provided through Project Independence 
and (on their own initiative) through non-Project Independence sources. The 24-month survey data 
afford the opportunity to observe participation over a longer follow-up period than does the 12-month 
program flow anal>sis. This is important because Project Independence's ongoing panicipation 
mandate was intended to require program group members to participate in employment-related 
activities for as long as they continued to receive AFDC payments and remained mandatory^ (and to 
the extent that resources permitted it). Since, as would be expected, some program group members 
engaged in employment-related activities outside of Project Independence during the follow-up period 
(usually after they had left the program), information about that panicipation is also included among 
the chapter's findings. 

Section III presents an analysis of the extent to which control group members gained access 
to (non-Project Independence) employment-related activities on their own. Unlike previous studies. 



^Kemple and Haimson, 1994, Chapters 3-5. 

^Individuals are no longer mandatory for Project Independence when they stop receiving AFDC. 
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however, this analysis also had to include an estimate of the extent to which, contrary to the research 
design, control group members were exposed to the Project Independence treatment by participating 
in Project Independence activities or by being told (incorrectly) that they were required to do so. This 
departure from the research design is unusual and affects the interpretation of the impact findings 
presented in Chapters 5 and 6. Nevertheless, since the great majority of control group members were 
not exposed to Project Independence, their experiences still provide the best benchmark for assessing 
whether Project Independence increased (relative to the control group) the program group's level of 
involvement in employment-related activities. 

The final section of the chapter makes this assessment. In other words, it examines Project 
Independence's net effect on participation in employment-related activities. The analysis uses new 
data, based on the 24-month survey, to compare the experiences of program and control group 
members over the two-year period after each member entered the research sample. It presents the 
differences between the program and control groups in participation rates and length of participation 
across particular types of activities. This is crucial because, even in cases where a high proportion 
of the program group received a particular service, if control group members were just as likely to 
receive the same service, then that service is unlikely to have contributed much to Project 
Independence's impact on employment or welfare receipt or its benefit-cost results (unless the quality 
of the services received by the program and control groups was substantially different). 

In brief, this chapter shows that virtually all of the program group members were exposed to 
Project Independence. By contrast, and contrary to the research design, an estimated 20 percent of 
the control group were exposed to Project Independence by attending a program orientation or (much 
less commonly) by participating in employment activities through the program. Even after accounting 
for the control group's exposure to Project Independence, however, the analysis shows that the 
program group did participate in employment-related activities to a substantially greater extent than 
the control group — in other words, that Project Independence had a large impact on the rate of 
participation on such activities. In all, 64 percent of the program group participated in at least one 
employment-related activity provided through Project Independence or non-Project Independence 
sources during the two-year follow-up period. This represents a 60 percent increase over the control 
group participation rate of 40 percent. Project Independence had its largest impact on participation 
in independent job search activities, more than doubling the control group's participation rate in tliis 
activity. The program had a smaller effect on the receipt of basic education - adult basic education 
(ABE), General Educational Development (GED) classes, which prepare people to take the GED test,^ 
and instruction in English as a Second Language (ESL); vocational training; and post-secondary 
education services in addition to vocational training. By combining a strategy of emphasizing 
independent job search over more expensive education and training with an attempt to engage a very 
large portion of the AFDC caseload for at least a brief spell in the program. Project Independence was 
operated as a JOBS program with a relatively low cost per person in the program. 

As noted earlier, most of the program group members who participated in Project 
Independence began their involvement prior to October 1992, when the program modified its job- 
readiness criteria. This modification enabled more participants to choose education or training as their 



-^Those who pass the GED test are given a certificate that is intended to signify knowledge of basic high 
school subjects. 
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first activity. The state and the local programs have continued to revise program operating strategies 
as they acquire more experience and respond to changing funding circumstances and caseload 
dynamics. Thus, the treatment received by program group members prior to October 1992 may not 
fully represent the current mode of operating Project Independence. 

L The Project Independence Treatment' * 

This section of the chapter is based on the first 12 months of follow-up. It begins by 
describing the employment-related services available to Project Independence participants and staff and 
participants* perceptions of the availability and quality of these services.^ It then examines the 
program group's ''flow" through the Project Independence program model during the initial 12-month 
follow-up period. 

A* The Nature of Project Independence Services 

A cornerstone of JOBS, and of the Florida Employment Opportunity Act of 1987, which 
created Project Independence, is an emphasis on enhancing coordination and cooperation among state 
agencies and other organizations to provide employment-related services for public assistance 
recipients. The primary purpose is to provide more - and more efficiently delivered - services to 
public assistance recipients than a single agency or organization could provide alone. Thus, a key 
function of Project Independence staff is to serve as conduits for linking program participants with 
necessary and appropriate services. 

The Staff Activities and Attitudes Survey asked Project Independence staff a series of questions 
concerning the availability and quality of key Project Independence services. The findings are 
presented in Table 3.1, which shows the proportion of staff who rated the normal availability of each 
service as high and the proportion who rated each service as worthwhile for those assigned to it.^ 
The table indicates that a high percentage of Project Independence staff surveyed from the nine 
research counties reported that services were normally available to program participants. However, 
staff varied somewhat in their assessment of the "quality" of those services. Overall, 80 percent of 
staff rated the availability of services as high, while 64 percent rated the services as worthwhile for 
participants. 

The 12-month survey of a subsample of program group memb^^rs also asked those who 
participated in these activities whether they thought they were helpful. These findings are also 
presented in Table 3.1, which shows the proportion of participants who rated each service as very 



"^This discussion is adapted from MDRC's first report on Project Independence (Kemple and Haimson. 
1994). 

^Tne employment-related services discussed in this report include independent job search, job club, basic 
education, and vocational training and post-secondary education. In some counties. Project Independence also 
provided work experience activities (jobs in public or non-public agencies that are required as a condition of 
receiving AFDC). The evaluation, however, did not find evidence that members of the research sample ever 
participated in work experience activities. 

^See Kemple and Haimson, 1994, Chapter 3, for a description of the measures used in Table 3. 1. 
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TABLE 3,1 

AVAILABILITY AND QUALITY OF PROJECT INDEPENDENCE SERVICES IN THE RESEARCH COUNTIES 



All Counties Bay Broward Dade Duval Hillsborough • Lee Orange Pinellas Volusia 
Variable „.„. . _ ' . . . (%) (%) (%) (%) (% ) (%)__(%) i%) (%) (%! 



Service a vaila bility 

Staff who rated availability of program 
services as high (a) 

All services (b) 

Job club 

Basic education (c) 

Training and connmunity college (d) 



80.4 


100.0 


84.2 


76.5 


82.1 


76.0 


100.0 


83.3 


77.8 


100.0 


82.1 


100.0 


94.1 


75.8 


92.9 


88.0 


66.7 


91.7 


58.8 


85.7 


82.4 


100.0 


89.5 


79.4 


74.1 


84.0 


100.0 


91.7 


76.5 


100.0 


75.8 


100.0 


73.7 


74.6 


81.5 


68.0 


66.7 


66.7 


77.8 


100.0 



Service quality 

Staff who rated program services 

as worthwhile for assigned participants (a) 

All services (b) 

Job club 

Basic education (c) 

Training and community college (d) 

Participants who rated program services 
as very helpful (e) 

All services (b) 

Independent job search 

Job club 

Basic education (c) 

Training and community colje^ejdl 



63.7 


25.0 


68.4 


66.7 


50.0 


70.8 


66.7 


33.3 


94.4 


42.9 


67.1 


25.0 


64.7 


70.8 


57.7 


79.2 


66.7 


50.0 


75.0 


66.7 


70.1 


50.0 


73.7 


72.7 


52.0 


83.3 


66.7 


58.3 


94.1 


28.6 


68.9 


75.0 


68.4 


69.7 


60.0 


66,7 


66.7 


54.6 


88.9 


71.4 


52.6 


63.0 


43.6 


50.0 


57.7 


51.3 


64,7 


37.0 


57.5 


70.8 


43.9 


56.0 


38.5 


39.8 


53.5 


33.9 


53.9 


40.0 


46.2 




69.9 




















64.9 




















78.8 





















SOURCES: The Staff Activities and Attitudes Survey and the 12-month survey of a subsampie of program group members. 



NOTES: Dashes indicate that the calculation was omitted because the sample size is less than 20. 

(a) The calculations are based on responses from Project Independence staff employed at the time the Staff Activities and Attitudes Sun/ey 
was administered (September-October 1991). They include supervisors. HRS employment counselors and interviewers, and LES employment specialists 
and representatives. They do not include HRS support staff. 

(b) Includes average responses to questions about job club, ABE, GED, ESL, training and vocational education, and community college services. 

(c) Includes average responses to questions about ABE. GED. and ESL sen/ices. 

(d) Includes average responses to questions about training, vocational education, and community college sen/ices. 

(e) The calculations are based on responses from the 12~month survey subsampie who participated in the respective services: therefore, the 
sample size varies for each program service. 
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helpful. Among participants, 53 percent rated Project Independence services as very helpful. This 
ranged from a low of 37 percent in Orange County to a high of 71 percent in Volusia County. 

These aggregate ratings across all Project Independence services mask differences in staff and 
participant perceptions of specific activities. These are discussed below in the context of a more 
'detailed description of the various activities. Then Section IB focuses on the rates at which program 
group members in the research sample participated in these activities within ^'i.e first 12 months 
following their initial referral to Project Independence. 

1. Independent Job search . During the Project Independence orientation, participants 
received a detailed explanation of Project Independence's opportunities and obligations. Case 
managers then conducted an initial assessment of their educational attainment and recent work 
experience to determine whether they were "ready" to enter the labor market immediately. Based on 
the outcome of this assessment, case managers referred participants to either up-front job search or 
a formal assessment of their education and training needs. Independent job search was generally the 
first activity assigned to job-ready participants and involved a two-week period during which 
participants looked for work on their own.^ A referral to independent job search usually occurred 
at the end of the Project Independence orientation, when job-ready participants were given forms on 
which to document their job contacts.^ Participants assigned to independent job search activities were 
required to make at least six in-person employment applications per week. In order to meet the 
participation requirement for these activities, they must have documented the applications on a job 
search report form and then had to return the form to their case manager at the end of each two-week 
period. MDRC's interviews with Project Independence staff indicated that the case managers did not 
often contact the employers listed on the forms to confirm that the participant had submitted a job 
application. However, case managers did tend to follow up with participants who did not turn in their 
forms at all, and in some cases this led to referrals for sanctioning. Case managers also encouraged 
participants to use ttie job referral services of Project Independence or LES-contracted staff who have 
access to Job Service employment contacts. These staff provided job leads from the Job Service job 
bank, gave advice on conducting a job search, and offered other support. In general, however, 
participants did not appear to have made extensive use of these services. 

As discussed later in the chapter, independent job search activities were the most widely used 
Project Independence services. This contrasts with many other welfare-to-work programs that also 
emphasized immediate job placement for large portions of the caseload, but did so through more 
intensive and expensive activities such as job clubs and supervised job search assistance.^ 

The 12-month survey of a subsample of program group members asked participants in job 
search about whether they thought Project Independence was helpful in putting them in contact with 
employers and telling them about job openings. As shown in Table 3.1, 44 percent of participants 



^In Project Independence, job search activities are often referred to as "individual," "up-front," or (for 
second and subsequent job search assignments) "extended" job search. Because of the relatively low level of 
formal supervision diat Project Independence participants received during the job search activities, this report 
uses the term "independent" job search to describe them. 

^HRS-ES Fom 4133. 

^See Gueron and Pauly, 1991. 
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rated job search as very helpful, the lowest percentage for any of the activities listed. 

2. Job club > Job club was a workshop and training session intended to teach participants 
better ways to search for, apply for, and keep a job.^^ Participants were supposed to be assigned 
to job club if they were unsuccessful in finding employment on their own or, occasionally, if they had 
completed education and training activities and needed to develop job search skills. Job clubs were 
generally scheduled for at least 20 hours per week for two to four weeks and were conducted in two 
phases. During the first phase, participants received classroom instruction in job-seeking strategies, 
procedures for completing a job application, job-interviewing skills, labor market trends, and life 
skills. During the second phase, participants were supposed to be involved in a supervised group job 
search in which they would receive instruction on how to make employer contacts and would schedule 
job interviews over the phone. During this group job search, participants were also intended to 
practice making phone calls to employers under the supervision of program staff. In most Project 
Independence units, job clubs were conducted by LES employment specialists. In some counties, 
however. Project Independence established a subcontract with the local Job Training Partnership Act 
(JTPA) agency to conduct their job clubs or referred participants to job clubs conducted by the JTPA 
agency. The top panel of Table 3.1 shows that 82 percent of Project Independence staff across the 
nine counties rated the availability of job club as high. This ranged from 59 percent in Pinellas to 100 
percent in Bay. 

MDRC's observations of several job clubs indicated some variation in the quality and emphasis 
of the classroom instruction and group job search components. For example, staff in one unit had 
developed innovative role-playing exercises to help participants practice interviewing skills, while staff 
in anotfier unit focused on helping participants contact employers and schedule interviews as quickly 
as possible. Some job clubs, however, appeared to place very little emphasis on the group job search 
components, in part because the units did not have adequate telephone service for participants to use 
to make employer contacts. In these units, job club was primarily a job-readiness workshop, with little 
emphasis on group-directed efforts to look for work. Participants were asked to develop job leads by 
looking at the want ads and usually contacted employers on their own, rather than in a group setting 
using a telephone room. 

The variable quality of job clubs is reflected in staff responses to the Staff Activities and 
Attitudes Survey. As shown in Table 3.1, 67 percent of all Project Independence staff rated job club 
services as worthwhile for participants. This ranged from 25 percent in Bay to 79 percent in 
Hillsborough. 

The 12-month survey of a subsample of program group members also asked job club 
participants ho^' much instructors helped them, how much job club increased their confidence about 
looking for a job, and how much the activity helped them learn useful job-hunting skills and 
techniques. Table 3.1 indicates that 70 percent of participants rated job club as very helpful. In 
general, participants were much more likely to give a rating of "helpful" to job club than to 
independent job search. 

3. Basic education services . Project Independence basic education services, as well as 



*^In Project Independence parlance, job club is known as "job-readiness skills training." 
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the training activities to be discussed shortly, were targeted to participants the program defined as not 
job-ready and to others who were not able to find work on their own. (As discussed in Chapter 2, 
only 10 percent of the Project Independence evaluation sample would have been defined as not job- 
ready at random assignment.) Basic education services included adult basic education (ABE), 
consisting of remedial classes designed to teach participants basic literacy and num.ijracy skills 
equivalent to the completion of the eighth grade; General Educational Development (GED) classes, 
which give participants an opportunity to prepare for the GED test; and English as a Second Language 
(ESL) classes, which offer intensive instruction in English to participants who are not proficient in it. 
Participants who were not job-ready were usually referred to basic education activities after a formal 
assessment of their education needs (including literacy testing) and the development of an 
Employability Plan. In most counties. Project Independence coordinated with local education agencies 
(LEAs) for basic education services. 

The findings presented in Table 3.1 indicate that more than 80 percent of Project Independence 
staff across the nine counties rated basic education services as nonaally available. This high 
percentage was consistent across all counties, suggesting that there were very few slot limitations in 
basic education services. 

Table 3.1 also indicates that staff and participant ratings of the quality of basic education 
services were relatively high compared to their ratings of the other services. Seventy percent of 
Project Independence staff across the nine counties rated basic education services as worthwhile for 
the participants whom they referred to those services, and 65 percent of those who participated in basic 
education activities rated them as very helpful. 

4. Vocational training and post-secondary education . Project Independence also 
offered other types of education and training services primarily for participants with a high school 
diploma or GED. Vocational training provided training in an occupational skill that was expected to 
lead to employment. Community college education was usually provided if the education program was 
consistent with the participant's Employability Plan and offered training in skills that lead to 
employment. Community college education was generally limited to one year, but exceptions could 
be granted if approved by the case manager, the unit supervisor, and district program staff. Finally, 
some Project Independence participants were referred to JTPA agencies to be placed in on-the-job 
training (OJT) positions. 

Vocational training and post-secondary education services were generally provided by a 
consortium of local school districts, JTPA agencies, and community colleges. In some counties, local 
public school systems and community colleges were able to develop special programs targeted 
specifically to Project Independence participants. Interviews with Project Independence staff, however, 
indicated that the relationship between Project Independence and local JTPA programs was somewhat 
more variable and, in some cases, quite strained. This may have resulted from the fact that the JTPA 
system placed a very high premium on perfoimance standards. These standards, and the financial 
incentives that accompanied them, often provided an incentive for local JTPA programs to serve those 
who were most likely to experience positive outcomes and thus enhance program performance levels. 

Table 3.1 indicates that 76 percent of Project Independence staff felt that training and post- 
secondary education services were normally available - a perception that was quite consistent across 
the counties. Also, Project Independence staff and participants had generally positive perceptions of 
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the quality of training and post-secondary education services. Almost 70 percent of Project 
Independence staff felt that these services were worthwhile, and nearly 80 percent of the participants 
surveyed felt that they were very helpful. 

B. The Program Group Flow Through Project Independence During the First 12 Months 
After Random Assignment s^ 

Implementation of the Project Independence model can be viewed as having three levels. The 
first level involves engaging mandatory participants in a program orientation, during which staff 
explain the requirements and opportunities of participating in Project Independence and then refer 
participants to an initial activity. The rates at which program group members attended orientation (and 
the program's treatment of those who did not attend) provide an assessment of the extent to which the 
initial Project Independence mandate was enforced. The second level of implementation involves 
engaging participants in specific employment-related activities. These participation rates provide 
insights into the types of strategies that were emphasized in helping participants find work or prepare 
for work. The third level entails the process of keeping those who remain on AFDC and mandatory 
for Project Independence engaged in the program on an ongoing basis. Each of these levels is 
discussed below. 

1. Level 1; Complying with the initial participation mandate . Figure 3.1 illustrates 
the flow of 100 typical program group members through the Project Independence program model 
during the 12 months following each person's referral to a program orientation - which was also 
when random assignment took place. The second box in the top row of the figure shows that 77 
percent of program group members attended Project Independence orientation within the 12 months 
following their initial referral from the Public Assistance Unit. The first box in the second row shows 
that just over half of the program group members who never attended orientation (12 out of 23 
program group members) were sent a formal "sanction" notice from Project Independence indicating 
that, since they had not complied with program rules and were deemed not to have had a "good cause" 
for their noncompliance, their AFDC grant was in the process of being reduced (i.e., they were 
"referred for sanction"). Finally, the vast majority of those who did not attend orientation and 



^^This section draws on Project Independence casefiles and the 12-month survey data for the 725 program 
group members in the 12-month program flow subsample (see Chapter 2). These data sources, rather than 
the 24-month survey data, provide the best information about the extent to which program group members 
were exposed to the various aspects of Project Independence. In Section II of this chapter, the 24-month data 
are used to examine the program group's participation in employment-related activities through Project 
Independence and non-Project Independence sources over two years. 

^^"Referrals" for sanction, rather than actual AFDC grant reductions that resulted from sanction requests, 
are emphasized in this report in part because the data collected from Project Independence casefiles may not 
have included a complete record of whether the sanctions were actually enforced (i.e., the AFDC grant was 
reduced). In many instances, case managers closed the individual's Project Independence case for other 
reasons before they received a notice from the Public Assistance Unit indicating that the sanction had been 
implemented. In other instances, the casefiles did not contain either a sanction enforcement notice or a formal 
case closure notice, even though a sanction might have been enforced. Based on information that was found 
in the casefiles, 5 percent of those who did not attend orientation had a sanction enforced. The 12-montli 
survey of a subsample of program group members provides some evidence that this is a reasonably good 

(continued...) 
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FIGURE 3.1 



THE FLOW THROUGH THE PROJECT INDEPENDENCE MODEL 
FOR 100 TYPICAL PROGRAM GROUP MEMBERS 
WITHIN ONE YEAR AFTER RANDOM ASSIGNMENT 



100 
Referred to a 
Project Independence 
Orientation from the 
Public Assistance Unit 



77 

Attended an Orientation 



23 

Did Not Attend 
Orientation 



Ever Referred for Sanction 
and Deregistered (a) 12 

Never Referred for Sanction 
and Deregistered (a) 9 

Never Referred for Sanction 
and Not Deregistered (a) 2 



43 

Participated in at Least 
One Activity 



Independent Job Search 
or Job Club Only 25 

Education or 
Training Only 1 1 

'ndependent Job Search/ 
Job Club and 

Education/Training 7 



34 

Did Not Participate 
in Post-Orientation Activities 

Deferred 7 

Ever Referred for Sanction 
and Deregistered (a) 4 

Never Referred for Sanction 
and Deregistered (a) 1 8 

Never Referred for Sanction 
and Not Deregistered (a) 5 



23 

Registered (a) 



Participating (b) 1 1 

Not Participating (b) 1 2 



20 

Deregistered (a) 
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16 

Ever Employed 



20 

Ever Employed 



16 

Ever Employed 



20 

Ever Employed 



SOURCES: The 12-month program flow subsample, Florida Unemployment Insurance records, and Florida AFDC records. 
NOTES: (n) "Registered" refers to the finding thai an individual's case was open at the end of the 12-month follow-up period. 
"Deregistered" refers to the finding that an individual's case was closed at the end of the 1 2-month follow-up period. 
Q (b) "Participating" refers to tho finding that an individual was actively participating in an independent job search, job club, basic 

^|^(]] education, training, or community college activity at the end of the 12-month follow-up period. 



105 



were not referred for sanction were ultimately deregistered from the program. In many cases, these 
individuals either were never approved for AFDC or left AFDC quickly. 

In sum, 98 percent of the program group either attended orientation or, if they did not, were 
referred for sanction or deregistered from the program by the end of the follow-up period. In other 
words, only 2 percent of the program group did not experience some formal involvement with Project 
Independence.^^ This is strong evidence that Project Independence staff placed a heavy emphasis on 
enforcing this initial participation requirement. 

2. Level 2; Participating in employment-related activities . The third box in the top 

row of Figure 3.1 shows that 43 percent of program group members participated^"* in at least one 
post-orientation activity during the initial 12-month follow-up period. This represents 56 percent of 
those who attended orientation and is in the range of participation rates reported in other studies of 
mandatory welfare-to-work programs. The second box in the middle of Figure 3.1 shows that 34 
percent of the program group attended orientation, but did not participate in employment-related 
activities. The vast majority (85 percent) of these were either temporarily deferred from participation, 
were subject to formal sanctioning procedures, or were deregistered from the program. In a modest 
percentage of cases (5 percent of the program group), however, it appears that case managers did not 
follow up with program group members after orientation, even though they remained registered for 
the program. 

Table 3,2 displays the rates of participation in each type of Project Independence employment- 
related activity, calculated in three different ways to facilitate comparisons with findings from other 
studies. The first column presents these rates for all program group members, including those who 
never attended orientation and those who attended orientation but never started an activity. This 
approach is helpful for understanding the extent to which the entire program group received particular 
kinds of services. The second column presents the participation rates for only those program group 



^'^(... continued) 

estimate of the overall rate at which sanctions resulted in actual AFDC grant reductions. Of those who 
reported that they did not attend orientation, 8 percent indicated that their AFDC grant had been reduced at 
somt point during the follow-up period for a failure to participate in Project Independence. 

^•^These are the individuals labeled "never referred for sanction and not deregistered" in the first box in 
the second row of the figure. 

^"^In this evaluation, "participation" is defined as attending an activity for at least one day, though most 
participants attended much longer than that. 

^^For example, Riccio et al., 1989, showed that, within six months following their referral from the 
Income Maintenance (AFDC) offices, 71 percent of GAIN registrants attended orientation and 34 percent 
participated in at least one activity. Using a longer follow-up period (11 months), Riccio and Friedlander 
(1992) reported that approximately 56 percent of those who attended GAIN orientation participated in at least 
one post-orientation activity. This is very similar to the comparable figure for Project Independence (43 out 
of 77 is 56 percent). Gueron and Pauly (1991) reported on 13 evaluations of welfare-to-work programs in 
which between 38 and 64 percent of orientation attenders took part in at least one employment-related activity 
within a year after being referred to the program. However, comparisons with GAIN and other programs 
should be interpreted with caution because these programs differed from Project Independence and from one 
another in scale, eligibilit)' requirements, and procedures for bringing mandatory registrants into the program. 
Programs also differed in terms of the types of services they emphasized. 
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TABLE 3.2 



RATES OF PARTICIPATION iN PROJECT INDEPENDENCE ACTIVITIES 
FOR ALL PROGRAM GROUP MEMBERS, ORIENTATION ATTENDERS. AND PARTICIPANTS 
WITHIN ONE YEAR AFTER RANDOM ASSIGNMENT 



Participation Measure 


All Program 
Group Members 

(%) 


Program Group 
Members Who 
Attended an 
Orientation 

(%) 


Program Group 
Members Who 
Participated in any 
Activity 

(%) 


Ever attended a Project 
Independence orientation 


77,2 


100,0 


100.0 


Ever participated in any activity, 
excluding orientation and 
formal assessment (a,b) 


42,9 


55,5 


100,0 


Ever participated in independent 
job search or job club (b) 

Independent job search 

Job club 


32,3 
31.0 
9,9 


41.8 
40,1 
12,8 


75,3 
72,2 
23.0 


Ever participated in education or training (b) 
ABE, GED, or ESL 

Vocational training or post -secondary education 
Self- Initiated activities (c) 


17,6 
6,4 
7.3 
4,9 


22,8 
8.3 
9,5 
6,4 


41,0 
14.9 
17.1 
11.5 


Ever participated in both Independent job 
search or job club and education or training 


7.0 


9.1 


16,3 


Sample size 


725 


555 


292 



SOURCES: The 12-month program flowsubsample. 



NOIES: Results are weighted by the full research sample In each county. 

(a) Includes self- initiated activities that were approved by Project Independence. 

(b) Individuals could participate In more than one activity during the follov;-up period; therefore, the 
distribution may exceed the category percentage. 

(c) Includes self- Initiated basic education, vocational training, and community college courses. 
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members who attended Project Independence orientation. This information is useful for comparing 
these results with results from studies of welfare-to-work programs that identified the research sample 
at orientation. The final column presents participation rates for only those program group members 
who ever participated in at least one Project Independence activity. These measures are useful for 
gauging the relative emphasis Project Independence placed on particular services. 

Table 3.2 shows that a total of 32 percent of the program group members participated in at 
least one independent job search or job club activity during the follow-up period. This represents 75 
percent of those who participated in at least one activity and reflects the labor force attachment focus 
of Project Independence. The table also shows that independent job search was the most widely used 
Project Independence activity: 31 percent of the program group participated in independent job search 
(representing 72 percent of those who participated in at least one activity). Also, a total of 10 percent 
of the program group participated in at least one job club (23 percent of those who participated in at 
least one activity). In most cases, this occurred after they completed an independent job search. 
However, MDRC interviews with Project Independence staff indicated that, partly as a relatively low- 
cost strategy for meeting participation and job placement goals and for managing their very large 
caseloads, they often referred participants to two or three independent job search activities before 
referring them to a job club. Many of these participants may have found work (at least temporarily) 
and others may have become disengaged from the program after several independent job search 
activities. This may account, in part, for the relatively low level of participation in activities such as 
job club, education, or training, which were intended for job-ready participants who could not find 
work on their own. 

Table 3.2 also shows that 18 percent of the program group members participated in an 
education or training activity. This represents 41 percent of those who participated in at least one 
activity and indicates that Project Independence was not solely focused on independent job search. 
Staff appeared to place at least a modest emphasis on helping participants enhance their employability 
through education and training. Interestingly, training and community college courses, rather than 
basic education, represented the most widely used education activities, with 7 percent of the program 
group (17 percent of those who participated in at least one activity) participating in them. Finally, 7 
percent of the program group (representing 16 percent of those who participated in at least one 
activity) participated in both independent job search or job club and education or training activities. 

In sum, the high rate of participation in independent job search activities reflects the labor force 
attachment emphasis of the Project Independence program model. However, Project Independence 
chose to pursue a relatively low-cost and unintensive labor force attachment strategy by referring 
participants to independent job search rather than to supervised group job club activities. At the same 
time, a substantial percentage of program group members also made use of education and training 
services, either as tlieir only activity or in combination with independent job search or job club. Basic 



As noted in Table 3.2, some members of the program group participated in "self-initiated" education 
or training. This refers to activities people enrolled in on their own (usually) prior to the time they were 
referred to Project Independence, and which Project Independence subsequently approved as satisfying their 
Project Independence panicipation requirement. 

-47- 



.1 . 
X U u 



education services were used far less than in other JOBS programs. ' 

3. Level 3; Meeting the ongoing participation mandate ^ While the findings presented 
above indicate that Project Independence succeeded in engaging a high percentage of mandatory 
participants in the initial stage of the program model (a program orientation and the first activity 
referral for those who remained in the program), it was less successful in enforcing the ongoing 
participation mandate. For example, program group members participated in Project Independence 
employment-related activities for an average of 1.6 months during the 12-month follow-up period. 
This represents less than one-quarter of the average number of months they were registered in the 
program (7.8 months). 

This point is also illustrated in Figure 3.2, which presents a month-by-month breakdown of 
the AFDC and Project Independence status of the program group during the first 12 months following 
random assignment. The sections of each bar in the figure represent mutually exclusive categories. 
Month 1 in the figure represents the month during which the sample member applied or was 
redetermined for AFDC eligibility, randomly assigned, and then referred to Project Independence 
orientation.^^ 

Figure 3.2 shows that about two-thirds of the program group members were not subject to the 
Project Independence participation mandate at the end of the 12-month follow-up period (37 percent 
were not receiving AFDC at that point and another 29 percent were deregistered from the program 
but were receiving AFDC). However, of those who were receiving AFDC and were registered in 
Project Independence, 70 percent were not engaged in employment-related activities. Specifically, 13 
percent of the program group (representing 38 percent of those on AFDC and registered for Project 
Independence) were unaccounted for, and an additional almost 1 1 percent (representing 32 percent of 
those on AFDC and registered for Project Independence) were either waiting to start an activity. 



^'Findings from MDRC's evaluation of California's GAIN program indicate that approximately half of 
those who were engaged in at least one GAIN activity participated in basic education. County-specific 
percentages found in the GAIN evaluation ranged between 35 and 72. See Riccio, Friedlander, and 
Freedman, 1994. 

^^During any given month, it was possible for an individual to have been receiving AFDC and to have 
been in more than one of the remaining categories. For example, an individual could have participated in 
independent job search for part of a month and then remain registered and monitored for the remainder of the 
month. Thus, in order to construct meaningful mutually exclusive categories, a hierarchy was developed to 
reflect individuals' level of engagement with the program. Those who were deregistered from the program 
during a given month (and received AFDC) are included in the "deregistered" category. Those who 
participated in an activity during a given month (and received AFDC) are included in the "participating" 
category even if they completed that activity during the month. Those who did not participate in an activity 
but had been temporarily deferred from participation, were referred to an activity and were waiting to start, 
or were referred for a sanction during a given month are included in the "not participating but monitored" 
category. Finally, those who had no record of participation or referral to an activity during a given month 
(but were registered and received AFDC) ai'e included in the "not participating and not monitored" category. 

^^Month 1 represents the month of random assignment, which, for many sample members, occun'ed before 
AFDC eligibility was determined. This is reflected in the high percentage of program group members not 
receiving AFDC in month 1 (81 percent) and the dramatic decline (to 19 percent) in month 3 as their 
applications were approved. 
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FIGURE 3.2 



MONTH-BY-MONTH AFDC AND PROJECT INDEPENDENCE STATUS 
FOR THE PROGRAM GROUP WITHIN ONE YEAR AFTER RANDOM ASSIGNMENT 
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38.6 



26x8 



38.4 



37.2 



On AFDC 

Participating in a post-orientation 
activity (a) 



Not participating in a post-orientation 
activity but being monitored (a,b) 

Not participating in a post-orientation 
activity and not being monitored, but 
still registered for Project Independence 
(a,b) 

Deregistered from 
Project independence 



Off AFDC 
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SOURCES: Those 12-month program flow subsample members for whom a Project Independence casefile was obtained and Florida AFDC records. 
NOTES: Results are weighted by the research sample In each county. 

(a) Post-orientation activities include independent job search, job club, basic education, training, and community college activities. 
Orientation, assessment, and case management appointments are not included. 

(b) Program group members were defined as being "monitored" if they were registered for Project Independence and were referred 
to an activity (but had not yet started it), were temporarily excused from participation (deferred), or were in the process of being 

rnferred for a sanction. 
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deferred, or in the process of being referred for a sanction. Finally, at the end of the follow-up 
period, 10 percent of the program group (representing 30 percent of those on AFDC and registered 
for Project Independence) were participating in employment-related activities. 

IL The 24-Month Survey Findings on the Program Groups s Participation in Employment- 
Related Activities 

Past research has shown that a complete accounting of welfare recipients' participation in 
employment-related activities - as well as the costs of providing those services - requires that it be 
measured over several years and that it include services provided by welfare-to-work programs as well 
as by other agencies in the community. This section of the chapter extends the previous analysis by 
examining data from the 24-month survey that include information about program group members' 
participation in Project Independence and non-Project Independence activities during a two-year follow- 
up period.^^ The findings are presented in Table 3.3. 

The top panel of Table 3.3 displays rates of participation in employment-related activities, as 
reported by program group members on the 24-month survey. This panel represents all such activities 
— those program group members identified as having been provided by or through Project 
Independence and those provided by or through some other source. It shows that 64 percent of the 
program group members reported participating in at least one employment-related activity during the 
two-year survey follow-up period. It also shows that 43 percent of the program group members 
reported participating in at least one independent job search or job club activity and that 42 percent 
reported participating in at least one education or training activity. 

The bottom panel of Table 3.3 shows that 16 percent of the program group members reported 
participating in education or training activities through Project Independence.^^ This means that 
approximately 26 percent of the program group (the difference between the overall rate of 42 percent 
and the Project Independence rate of 16 percent) participated in education or training activities through 
non-Project Independence sources. Thi:; accounts for over 60 percent of all participation in education 
or training activities reported by program group members on the 24-month survey. In contrast. 



Appendix D presents findings from an analysis that projects participation panems over a five-year 
period. These are used in the longer-term cost projections presented at the end of Chapter 4, which form the 
basis of the benefit-cost analysis presented in Chapter 7. 

^^It is important to note that the 24-month survey findings on Project Independence participation patterns 
are generally consistent with the findings from the 12-month program flow analysis discussed above. Both 
data sources reflect the relative emphasis Project Independence placed on independent job search and job club 
activities compared to education and training activities. However, the self-reported (i.e., survey-reported) 
rates of participation in Project Independence education and training activities over the two-year survey period 
are actually slightly lower than those found by analyzing the 12-month casefile data. This is due. in large 
part, to the fact that survey respondents were not likely to report self-initiated education and training activities 
as being part of their involvement with Project Independence. Since, as noted earlier, such activities usually 
began prior to an individual's involvement with Project Independence, they were especially likely to be seen 
from a client's perspective as identified with the local community college, vocational training school, or 
community-based organization the client attended. In other words, such activities would be greatly under- 
counted in the bouom panel of Table 3.3, but would be fully counted in the top panel. 
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TABLE 3.3 



RATES OF PARTICIPATION IN PROJECT INDEPENDENCE AND 
NON-PROJECT INDEPENDENCE EMPLOYMENT-RELATED ACTIVITIES 
BY PROGRAM GROUP MEMBERS WITHIN TWO YEARS AFTER RANDOM ASSIGNMENT 



Program 
Group 



Activity Source and Participation Measure (%) 

Project Independence and 

non-Project Independence sources combined 

Ever participated in any employment- 
related activity (a) 63.9 

Ever participated in independent 

job search or job club (a) 42.7 

Independent job search 41 .0 

Job club 13.2 

Ever participated in education or training (a.b) 41 .5 

ABEorGED 12.1 

ESL 4.6 

Vocational training or post-secondary education 29.2 

On-the-job training 4.0 

Project Independence activities 

Ever participated in any employment - 

related activity (a) 44.4 
Ever participated in independent 

job search or job club (a) 37.5 

Independent job search 36.0 

Job club 1 1 .6 

Ever participated in education or training (a.c) 16.4 

ABE or GED 5.6 

ESL 2.7 

Vocational training or post -secondary education 9.4 

On-the-job training 2.5 



Sample size 520 



SOURCE: The 24 -month survey of asubsample of program and control group members. 

NOTES: (a) Individuals could participate in more than one activity during the follow-up period; 
therefore, the sum of percentages in specific activities may exceed the category percentage. 

(b) This includes self- initiated basic education, vocational training, or post-secondary 

education. 

(c) These participation rates are slightly lower than those presented in the first Project 
Independence report. This is probably because some respondents to the 24-month survey 
did not identify their self-initiated education and training activities as Project Independence 
activities. In this study, self- initiated activities that Project Independence allowed to continue as 
satisfying the program's participation requirements are categorized as Project Independence 
activities. 

-51" 



X ^ ^ 



program group members attributed virtually all of their participation in independent job search or job 
club activities to Project Independence. This is not surprising, since education and training activities 
were much more widely available through other sources than were job search or job club activities. 

III. Participation in Employment-Related Activitie s Amon p Control Group Members During 
the Two Years After Random Assignment 

As discussed in Chapter 1, in a random assignment evaluation of a welfare-to-work program, 
the behavior of a randomly selected control group (which is not permitted access to the program under 
study) is the bencbuark against which to measure the behavior (including the participation rates) of 
the (also randomly assigned) program group. The difference between the two groups' behavior over 
time represents the effects (or "impacts") of the program being evaluated. 

This section of the chapter assesses the extent to which control group members in the Project 
Independence evaluation participated in employment-related activities. This is important because, as 
shown by previous random assignment evaluations of welfare-to-work programs, many AFDC 
applicants and recipients participate in employment-related activities even when they are not required 
to do so and even when they do not have access to case management and other services provided by 
programs such as Project Independence.^^ By definition, tlie higher the participation rates of the 
control group (relative to those of the program group), the smaller the effects of the program on 
participation rates. 

Four important factors provide a critical context for this discussion. First, as discussed earlier, 
all control group members were to be informed that they were not permitted (or required) to participate 
in Project Independence for a period of two years following their random assignment. Second, all 
control group members were also to be told that they could apply for other employment-related 
activities on their own initiative but were not required to do so. To facilitate their access to these 
services, they were provided with the names, addresses, and phone numbers of JTPA programs, public 
schools and community colleges, and community-based organizations that provide employment-related 
services for low-income populations. Third, all control group members were told that they would be 
eligible to receive subsidized child care and college tuition assistance under the same priorities and 
guidelines as those applicable to Project Independence participants.^^ Fourth, and contrary to the 
research design, some control group members were permitted to attend a Project Independence 
orientation, and some of these individuals went on to participate in employment-related activities 
through the program. 



'^^See Riccio, Friedlander, and Freedman. 1994; Friedlander and Hamilton, 1993; and Gueron and Pauly, 
1991. 

'^^Control group members were told that they could receive tuition assistance through HRS if they were 
enrolled in a community college program that would have been approved by Project Independence if these 
individuals had not been in the control group. They were also provided with a list of local child care agencies 
and informed that they could receive subsidized child care if they were enrolled in an activity that would have 
been approved by Project Independence if they had not been in the control group, and if child care resources 
were available. These modifications to the research design were made in response to recommendations of tiie 
Project Independence Evaluation Advisory Panel and the Florida State Legislamre, which were concerned thai 
control group members might be unfairly harmed by being excluded from Project Independence. 
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The first three factors were in keeping with the research design that was developed by MDRC 
in collaboration with HRS. They account for a large portion of the control group's participation in 
employment-related activities. The fourth factor represents a departure from the research design. The 
second part of this section, therefore, provides an assessment of the extent to which exposure to 
Project Independence may have affected the control group's overall rates of participation in 
employment-related activities. 

A. Control Group Participation in Employment-Related Activities 

The top panel of Table 3.4 displays rates of participation in employment-related activities, as 
reported by control group members on the 24-month survey. It shows that 40 percent of the control 
group members reported participating in at least one employment-related activity during the two-year 
survey follow-up period. It also shows that 19 percent of the control group members reported 
participating in at least one independent job search or job club activity and that 31 percent reported 
participating in at least one education or training activity. In particular, control group members were 
most likely to report participating in vocational training or post-secondary education activities (21 
percent), followed by independent job search (18 percent). Again, the higher rate of participation in 
education and training activities (compared with the rates for independent job search or job club 
activities) can be attributed, in large part, to the greater availability of these services outside Project 
Independence. 

Findings from previous evaluations of mandatory welfare-to-work programs provide a context 
for assessing whether the participation rates for control group members are unusually high. In the few 
cases where data on the control group's receipt of services are available, these evaluations found 
substantial participation in education and training activities, but very little if any participation in job 
search or job club activities.^"* These findings suggest that the control group's rate of participation 
in education and training activities in the Project Independence evaluation is consistent with the rates 
found in other evaluations. However, the Project Independence control group participated in 
independent job search activities to an unusually large extent. Several factors may account for this. 
First, the 24-month survey used in the Project Independence evaluation asked control group members 
specifically about independent job search activities in which they contacted employers and submitted 
job applications on their own and without the help of a counselor or a special class or the support of 



^ For example, in the GAIN evaluation, over 23 percent of control group members participated in some 
type of education or training activity within two to three years after random assignment, but only 4 percent 
participated in job search activities (see Riccio, Friedlander, and Freedman, 1994), Similarly, the evaluation 
of the San Diego Saturation Work Initiative Model (SWIM) Program found that over 28 percent of control 
group members participated in education or training activities within two to three years after random 
assignment, but that less than 1 percent participated in job search activities (see Hamilton and Friedlander, 
1989). Two other evaluations from the 1980s - Virginia's Employment Services Program and Illinois' Work 
Incentive (WIN) Demonstration Program in Cook County — provide findings on participation in employment- 
related activities for control group members. Thirteen percent of the control group in the Virginia evaluation 
participated in education or training activities within 15 to 28 months after random assignment; rates of 
participation in job search activities were not reported for control group members. In the Cook County 
evaluation, 18 percent of the control group members were active in education or training activities within nine 
months after random assignment, and just over 2 percent participated in job search activities. 
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TABLE 3.4 



RATES OF PARTICIPATION IN PROJECT INDEPENDENCE AND 
NON-PROJECT INDEPENDENCE EMPLOYMENT- RELATED ACTIVITIES 
BY CONTROL GROUP MEMBERS WITHIN TWO YEARS AFTER RANDOM ASSIGNMENT 



Control 
Group 

Activitv Source and Participation Measure (%) 

Project Independence and 

non -Project Independence sources combined (a) 

Ever participated in any employment- 
related activity (b) 40.1 

Ever participated In independent 

job search or job club (b) 1 8.7 

Independent job search 1 7.9 

Job club 5.8 

Ever participated in education or training (b) 30.5 

ABE or GEO 8.3 

ESL 3.0 

Vocational training or post— secondary education 21 .4 

On-the-job training 1 

Project Independence activities (c) 

Ever participated in any employment- 
related activity (b) 7.5 

Ever participated in independent 

jobsearchor job club (b) 6.0 

Independent job search 5.8 

Job club 1.9 

Ever participated in education or training (b) 2.2 

ABE or GED 0.7 

ESL 0.1 

Vocational training or post -secondary education 1 .4 

On-the-job training 0.3 



Sample size 509 



SOURCES; The 24 -month survey of a subsample of program and control group members and 
Project Independence casefile data, 

NOTES: (a) These percentages were calculated from the 24 -month survey data. 

(b) Individuals could participate in more than one activity during the follow-up period, 
therefore, the sum of percentages in specific activities may exceed the category percentage. 

(c) These percentages were calculated from Project Independence casefile data. 
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a group of similarly situated AFDC recipients. Project Independence provided independent job search 
activities for some panicipants, but this is also the kind of job search activity required of those who 
receive Unemployment Insurance or who leave AFDC but continue to receive Food Stamps.^- It is 
possible that some control group members were subject to the job search requirements of 
Unemployment Insurance or Food Stamp programs at some point during the two-year follow-up 
period. Second, programs such as those operated through JTPA and the Job Service of Florida offer 
job search services for all AFDC recipients. It is possible that control group members obtained these 
services on their own initiative. Third, some education and training programs offer job search services 
as part of occupational skills or post-secondary education classes. Some control group members may 
have received these services in conjunction with their participation in these activities. 

Finally, and most significantly, the relatively high rate of participation in employment-related 
activities, particularly independent job search activities, is due, in part, to the finding that some control 
group members gained access to Project Independence. This issue is addressed in greater detail below. 

B. Control Group Participation in Project Independence 

This section of the chapter examines three components of the control group's exposure to 
Project Independence: (1) being subject to the participation mandate by being referred for an AFDC 
gram reduction (sanction) for failure to attend a Project Independence orientation; (2) attending a 
Project Independence orientation; and (3) participating in employment-related activities through Project 
Independence. Using data collected from Project Independence casefiles, it was estimated that 19.8 
percent of the control group were exposed to at least one of these three components during the two- 
year follow-up period.^^ The three components of the control group's exposure to Project 
Independence are discussed below. 

First, the Project Independence casefile data indicate that, during the two-year follow-up 
period, an estimated 1.2 percent of the control group members were referred for a sanction for not 
attending a program orientation for which they were (incorrectly) scheduled. It is not known whether 
these sanction referrals resulted in actual grant reductions. It should be noted, however, that none of 



"^AIl recipients of Unemployment Insurance are required to look for work and to provide evidence of their 
job applications on a monthly basis. Similar requirements are sometimes placed on Food Stamp recipients 
who do not also receive AFDC. In Florida, this requirement for Food Stamp recipients is also administered 
through Project Independence, although very little funding is available and the program has relatively few 
participants. 

^^Estimating the control group's exposure to Project Independence was complicated by limitations in the 
24-month survey data. By comparing survey responses against Project Independence casefiles. it became 
apparent that the 24-month survey did not provide an accurate reflection of the extent to which the control 
group's participation in employment-related activities was associated with Project Independence. Most likely, 
this occurred because several survey questions that were mtended to be about Project Independence may have 
been construed by control group respondents as being about other employment and training programs. As 
noted earlier in this chapter, however, findings on the program group's participation in Project Independence 
activities from the 24-month survey are generally consistent with findings from the Project Independence 
casefile data that were used for the 12-month program flow analysis. A more detailed discussion of the data 
issues and analysis strategy used to estimate the control group's involvement in Project Independence is 
presented in Appendix C. 
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these control group members attended an orientation or participated in employment-related activities 
through Project Independence during the follow-up period. However, control group members may 
have changed their behavior in other ways in response to the threat of an AFDC grant reduction. 

Second, using data collected from Project Independence casefiles, it was estimated that 
approximately 1 L 1 percent of the control group members attended a Project Independence orientation 
but did not participate in any employment-related activities through the program after that point. It 
is not clear how much this type of exposure to the program affected the behavior of control group 
members. MDRC's observation of Project Independence orientation sessions and interviews with 
Project Independence staff indicate that the orientations were sometimes occasions when control group 
members were identified and prohibited from further participation. However, after hearing about the 
participation requirements, control group members (assuming that they were, in fact, treated as 
mandatory) may have been prompted to find work and leave AFDC more quickly than they would 
have otherwise. Thus, the 11,1 percent attendance rate may represent an upper bound on the most 
intensive exposure the control group could have experienced through orientation. 

Third, and probably constituting the most intensive involvement the control group had with 
Project Independence during the follow-up period, an estimated 7.5 percent of the control group 
members participated in at least one employment-related activity through Project Independence during 
the two-year follow-up period (see the bottom panel of Table 3.4). This represents approximately 19 
percent of the control group's total participation in employment-related activities (7.5 of the 40.1 
percent presented in the top panel of Table 3.4). Table 3.4 also shows that 6.0 percent of the control 
group participated in independent job search or job club activities through Project Independence and 
that 2.2 percent participated in education and training activities through the program. This means that 
32 percent of the control group's participation in independent job search or job club activities occurred 
through Project Independence (6.0 of the 18.7 percent presented in the top panel of Table 3.4), 
compared to only 7 percent of their participation in education or training activities (2.2 of the 30.5 
percent presented in the top panel of Table 3.4). This reflects the job search focus of Project 
Independence and the fact ttiat such services are less readily available outside Project Independence. 

The sum of these three components - 1.2 plus li.l plus 7.5 - yields the estimate of 19.8 
percent of the control group who were exposed to some aspect of the Project Independence 
treatment. Chapter 5 discusses the implications of this exposure for interpreting the estimates of 
Project Independence's impacts on earnings, employment, and welfare receipt. As discussed earlier 
in this chapter, virtually the entire program group was exposed to Project Independence. Also, even 



'^^This is the most reliable estimate of the control group's exposure to Project Independence. In contrast, 
the 24-month survey data suggest that up to 44.8 percent of the control group may have been exposed to some 
aspect of the Project Independence treatment. This estimate is almost certainly much too high, given the types 
of confusion control group members may have encountered in answering questions about Project Independence 
on the survey. This estimate was obtained by combining: the 21 .8 percent of the control group who reported 
participating in employment-related activities through Project Independence; the 14 percent who reported only 
attending an orientation; the 1 percent who said they were sanctioned for not attending an employment or 
training program they were required to attend in order to receive AFDC; and the 8.1 percent who said they 
were informed that they were required to participate in an education, training, or employment program in 
order to receive AFDC. See Appendix C for a discussion of these estimates. 
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after the control group's exposure to Project Independence was factored in, the program was found 
to have substantially increased the program group's participation in employment-related activities 
relative to the control group (as will be shown in the next section of this chapter). Program-control 
group differences in employment, earnings, and AFDC receipt reflect the impact of these increases. 

Finally, it should be noted that most of the control group's exposure to Project Independence 
occurred during the first year of follow-up, and that the level of exposure was similar for the key 
subgroups analyzed in this report. Thus, the control group's erroneous involvement with Project 
Independence is not likely to have had an effect on the time pattern of program impacts discussed in 
Chapter 5 or on the subgroup differences discussed in Chapter 6. 

IV. Program-Control Group Differences in Participation During the Two Years After Random 
Assigmnent 

Using the 24-month survey data, this section of the chapter assesses the extent to which the 
Project Independence participation requirements and service delivery strategies had an impact on 
program group members' participation in employment-related activities. It does so by examining the 
difference between the program and control groups' rates and length of participation in employment- 
related activities. This section also examines Project Independence's effects on program group 
members' educational attainment (i.e., receipt of a credential), 

A. Program-Control Group Differences in Rates of Participation in Employment-Related 
Activities 

Table 3.5 directly compares the percentage of program and control group members who 
participated in employment-related activities within the two-year survey follow-up period. The 
difference in these rates represents the "impact" (i.e., effect) of Project Independence and indicates 
how much program group members' participation was affected by their involvement with the program. 
The table shows that 64 percent of the program group members participated in at least one 
employment- related activity during the survey period, compared to 40 percent of control group 
members — a 24 percentage point (approximately 60 percent) difference. The table also shows that 
Project Independence had its largest impact on participation in independent job search and job club 
activities (24 percentage points), with a program group rate more than double that of the control group 
(43 versus 19 percent). The program had a smaller, although still substantial, impact on participation 
in education and training activities ~ 11 percentage points, with the program group rate (42 percent) 
being about one-third higher than the control group rate (31 percent). 

In addition (not shown in the table). Project Independence more than doubled the rate at which 
program group members participated in both independent job search or job club activities and 
education or training activities (20 percent of the program group compared to 9 percent of the control 
group). Both groups were equally likely to engage in education or training activities without also 
participating in independent job search or job club. This means that Project Independence had its 
primary effect on participation by increasing the rate at which program group members engaged in 
independent job search or job club either as their only employment-related activity or in combination 
with education or training activities. 
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TABLE 3.5 



TWO-YEAR IMPACTS ON RATES OF PARTICIPATION IN PROJECT INDEPENDENCE 
AND NON -PROJECT INDEPENDENCE EMPLOYMENT-RELATED ACTIVITIES 



Program Control 
Group Group Percentage 
Activity Source and Participation Measure (%) {%) Difference Change 

Project Independence and 

non -Project Independence sources combined 

Ever participated in any employment- 



related activity (a) 


63.9 


40.1 


23.8 


59.3% 


Ever participated in independent 










job search or job club (a) 


42.7 


18.7 


24.0 


128.8% 


Independe. it job search 


41.0 


17.9 


23.1 


128.8% 


Job club 


13.2 


5.8 


7.4 


128.8% 


Ever participated in education or training (a) 


41.5 


30.5 


11.1 


36.4% 


ABE or GED 


12.1 


8.3 


3.9 


46.9% 


ESL 


4.6 


3.0 


1.7 


56.6% 


Vocational training or post -secondary education 


29.2 


21.4 


7.8 


36.5% 


On-the-job training 


4.0 


3.1 


0.9 


28.7% 


Sample size (total = 1 ,029) 


520 


509 







SOURCES; The 24-month survey of a subsample of program and control group members and Project Independence 
casefiles 



NOTES: Tests of statistical significance of differences between research groups were not performed. 

Rounding may cause slight discrepancies in calculating sums, averages, and differences. 

(a) Individuals could participate in more than one activity during the follow-up period; therefore, the sum of 
percentages in specific activities may exceed the category percentage. 
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B* Program-Control Group Differences in Length of Participation in Employment 
Related Activities 



Another aspect of whether employment-related activities are likely to make a difference for 
welfare recipients is the "amount" or intensity of their involvement in these activities. A key measure 
of intensity is the length of time sample members participated in employment-related activities. In this 
study, as in calculations of participation measures for federal reporting on JOBS programs, length of 
participation is defined as the average number of months in which a person participated for at least one 
day.^^ The top panel of Table 3.6 presents the estimated average length of participation for 24-month 
survey subsample members who started a specified activity during the two-year survey period .^"^ As 
indicated in the first row of the table, program and control group members who participated in the 
various activities were engaged in them for the same number of months — an average of 6.7 months 
during the survey period. With the exception of ESL classes, program group members who 
participated in education or training classes were engaged in these activities slightly longer than were 
control group members in these activities. 

More importantly, as shown in the top panel of Table 3.6, people in both groups participated 
in education and training activities more than three times longer than they did in independent job 
search or job club activities — 9 months compared to 2.5 months, with respect to the program group, 
with a range of 1.6 months in job club to 8.8 months in vocational training or post-secondary 
education. This is important because people in education or training activities are expected to take 
longer to enter the labor market than those in independent job search or job club activities. To ±e 
extent that Project Independence resulted in an increase in participation in education and training 
activities (relative to the control group), the program's effects on employment and further training may 
be delayed. 

The primary measure used to capture Project Independence's impact on levels of service 
receipt is the length of stay in activities across all program group members and across all control 
group members in the 24-month survey subsample. The bottom panel of Table 3.6 presents this 
information (including zeros for people who never participated in a given activity). When estimated 
this way, the results for program and control group members can be compared directly to determine 
Project Independence's impact on the use of employment-related services - an impact that will affect 
the program-control group differences in costs and, in turn, the program's overall cost-effectiveness. 
It shows that Project Independence increased the length of participation in all activities except ESL and 
on-the-job training. Again the largest impacts were for independent job search or job club, with 
Project Independence doubling the length of participation (from an average of .5 months for the control 
group to an average of 1.1 months for the program group). Project Independence also increased 
participation in vocational training and post-secondary education by over 50 percent (from an average 
of 1.7 months for the control group to an average of 2.6 months for the program group). 



This definition was used to be consistent with the general approach that was considered most appropriate 
for measuring participation for the cost analysis presented in Chapter 4. For individuals who participated in 
an activity for more than one spell, the length of participation measure reflects the sum of all spells in the 
activity during the two-year survey period. 

^^ecause the calculations in the top panel of Table 3.6 do not include all program and control group 
members in the 24-month survey subsample, the differences in the third column of the table do not represent 
program impacts. 
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TABLE 3.6 



TWO-YEAR IMPACTS ON THE AVERAGE NUMBER OF MONTHS OF 
PARTICIPATION IN EMPLOYMENT- RELATED ACTIVITIES 



Sample and Participation Measure 



Progranri 
Group 



Control 
Group 



Difference 



Percentage 
Change 



Subsample members 

who started the specified activity 

within two years after random assignment (a) 

Average number of months of participation 

Average number of months participating in 
independent job search or job club 

Independent job search 

Job club 

Average number of months participating in 
education or training 

ABE or GED 

ESL 

Vocational training or post -secondary education 
On-the-job training 



6,7 



2,5 
2,2 
1,6 



9,0 
5,9 
5,7 
8,8 
6,1 



6,7 



2,5 
2,4 
1.7 



8,2 
5,4 
8.9 
7,9 
5,5 



0,1 



0,0 
-0.1 
-0.0 



0.8 
0.6 
'3,2 
0.9 
0,6 



1 .5% 



0.4% 
-5.1% 
-1.2% 



9.8% 
10.6% 
-35.9% 
1 1 .4% 
10.5% 



Full subsample (b) 

Average number of months of participation 

Average number of months participating in 
independent job search or job club 

Independent job search 

Job club 

Average number of months participating in 
education or training 

ABE or GED 

ESL 

Vocational training or post -secondary education 
On-the-job training 



4,3 



1,1 
0.9 
0,2 



3,7 
0.7 
0,3 
2,6 
0,2 



2,7 



0,5 
0.4 
0.1 



2,5 
0.4 
0,3 
1,7 
0.2 



1.6 



0.6 
0,5 
0,1 



1.2 
0.3 
0,0 
0,9 
0,1 



55.1% 



129.7% 
116.1% 
124.8% 



49.8% 
62.5% 
0.3% 
52.1% 
42,1% 



S ample size (total = 1 ,029) 



520 



509 



SOURCE: The 24-month survey of a subsample of program and control group members. 

NOTES: Tests of statistical significance of the differences between research groups were not performed. 
Rounding may cause slight discrepancies in calculating sums, averages, and differences. 
Length of participation is defined as the number of months with any participation in a given activity. 

(a) Estimates in italics were based only on individuals who participated in the specified activities. Because 
some program and control group members were excluded from the calculations, the program -control group differences 
do not reflect the impact of Project Independence. 

(b) The results shown in the bottom panel were calculated by multiplying the average number of months 
participating in a given activity within two years after random assignment for those who started the activity by the 
percentage who reported participating in the activity. 
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In summary, Tables 3.5 and 3.6 indicate that Project Independence produced a substantial 
increase in participation rates and length of participation in employment-related activities for program 
group members. This impact was greatest for independent job search, the least costly and most widely 
used activity in Project Independence. Project Independence also increased the program group's 
participation in vocational training, post- secondary education, and ABE and/or GED classes. Impacts 
were smallest for ESL and on-the-job training, the least frequently used services by both program and 
control group members. These patterns are important because they are the key determinants of the 
cost estimates presented in Chapter 4. 

C. Program-Control Group Differences in Educational Attaiament 

An important outcome of the education and training activities discussed above is often the 
attainment of a credential (such as a General Educational Development — GED — certificate) or 
license that provides employers with an indication of competency or skill development in a particular 
area. One question for the Project Independence evaluation is whether the increase in participation 
in education and training activities resulted in an increase in educational attainment. 

Table 3.7 shows the percentage of program and control group members who reported that they 
obtained various education credentials during the two-year survey follow-up period. It indicates that 
Project Independence had very little impact on receipt of GEDs or high school diplomas, trade 
certificates, and associate's degrees. The final item in the table also shows that approximately 57 
percent of both the program and control groups had GEDs or high school diplomas at the end of the 
two-survey period, and that there were only small differences between the two groups in the 
percentage who had trade certificates and higher education degrees. 
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TABLE 3.7 



TWO-YEAR IMPACTS OF PROJECT INDEPENDENCE ON 
RECEIPT OF AN EDUCATION CREDENTIAL 



Outcome 



Program 
Group 
(%) 



Control 
Group 



Difference 



Percentage 
Change 



Received a GEO or high 
school diploma during 
the folbw-up period 

Received a trade certificate 
during the foliow-up period 

Received an associate's degree 
during the follow-up period 

Received a bachelor's degree 
during the follow-up period 

Had a credential by the end 
of the folbw-up period (a) (%) 

GED or high school diploma 

Trade certificate 

Associate's degree 

Bachelor's degree 



1.7 



8.6 



2.7 



0,2 



57,5 
31.9 
4.4 
2.2 



1.0 



6.3 



1.4 



0.4 



56,2 
31,6 
3,6 
1,9 



0.7 



2.3 



1,4 



-0,2 



1.3 
0.3 
0.8 
0.4 



68.0% 



36.6% 



101.5% 



-51,3% 



2.3% 
0,9% 
22,6% 
18,7% 



Sample size (total = 1 .029) 



520 



509 



SOURCE: The 24-month survey of a subsample of program and control group members, 

NOTES: Rounding may cause slight discrepancies in calculating sums, averages, and differences. 
Results are regression-adjusted. 

A two-tailed t-test was applied to the differences between the experimental and 
control groups. Statistical significance levels are indicated as *** = 1 percent; ** = 5 percent; 
* = 10 percent 

(a) This includes those who already had the credential at the time they entered the study and 
were randomly assigned, as well as those who received the credential at some point during the vvo-year 
follow-up period. 
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CHAPTER 4 



THE COST OF PROJECT INDEPENDENCE AND 
NON^PROJECT INDEPENDENCE ACTIVITIES 



The cost analysis presented in this chapter estimates how much the government spent on Project 
Independence per program group member. It also shows how these costs varied across program 
components and support services. In so doing, it separates expenditures made by the Department of 
Health and Rehabilitative Services (HRS) - for administration, case management, and other direct 
services - from those made by other agencies providing education and training activities for Project 
Independence participants. This information may be useful to administrators and planners who want 
to understand, in a comprehensive way, the nature of the government's investment in Project 
Independence. It may be especially useful to those who may wish to alter the structure and operation 
of Project Independence, or who may need to estimate the cost of other state and national welfare 
reform proposals. 

However, the primary goal of the cost analysis is to estimate the government's average net cost 
of providing employment-related services to members of the program group. The net cost is the 
difference between the average cost per program group member and.the average cost per control group 
member of all Project Independence and non-Project Independence employment-related services that 
were used during a fixed period of time following a person's entry into the study. ^ In the benefit-cost 
analysis (Chapter 7), to assess whether Project Independence has been a cost-effective program from 
the perspective of government budgets, the net cost will be compared to the value of any budgetary 
savings during the same period (e.g., through lower average payments for AFDC, Food Stamps, and 
Medicaid) and tax revenue increases associated with additional earnings of program group members. 
Chapter 7 also analyzes the benefits and costs from the perspectives of the welfare clients and society 
as a whole. 

This chapter presents costs for the two-year period following each sample member's random 
assignment (the point at which program group members were referred to Project Independence). A 
five-year cost estimate is also calculated; it adds projected costs for the subsequent three years to the 
two-year estimates. The five-year cost estimate is less certain than the two-year estimate because of 
data limitations, but it attempts to capture the costs of additional participation that is likely to have 
occurred in years 3 to 5 and that, consequently, may have contributed to any impacts of Project 
Independence during these later years (which are included in the benefit-cost analysis). 

To summarize the main findings presented in this chapter: The estimated average cost (in 1993 
dollars) of Project Independence per program group member within two years after random assignment 
was $921. About 47 percent of this ($430) was spent directly by HRS, with the remainder ($491) 
having been spent by schools and other agencies, mostly in providing education and training to Project 
Independence participants. The non-Project Independence cost per program group member (usually 



^As discussed in Chapter 3, and contrary to the research design, an estimated 20 percent of controls 
received some type of Project Independence service. The cost of these services is included in the average cost 
per control group member. 
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for activities they entered after leaving the program) added another $1,041, bringing the total cost per 
program group member to $1,962. Comparing this cost to the total cost of all Project Independence 
and non-Project Independence services per control group member during the same period (i.e., $1,074) 
yielded a two-year net cost of $888 per person in the program group. The estimated five-year net cost 
is $1,150 per program group member, or $262 higher than the two-year estimate. 

The chapter commences with an explanation of the major components of the cost analysis. 
Sections II through V discuss the two-year cost findings for the program group, while Section VI does 
the same for controls. Net costs for the two-year period are presented in Section VII. Finally, Section 
VIII presents the five-year cost estimates. 



L Main Components of the Cost Analysis^ 

Figure 4.1 illustrates the main expenditure components for both the program group and the 
control group. For the program group, it shows that Project Independence-related expenditures by 
HRS (box IP) plus Project Independence-related expenditures by agencies other than HRS (box 2P) 
constituted the total cost of Project Independence (box 3P). When wiz-Project Independence 
expenditures (box 4P) are added to the Project Independence expenditures, the result is the total gross 
cost per program group member (box 5P). This was the government's total investment per person in 
the program group. 

Boxes IC to 5C in Figure 4.1 depict the types of expenditures per control that were included 
in this analysis. The Project Independence-related expenditures by HRS (box IC) and the Project 
Independence-related services by agencies other than HRS (box 2C), which together made up the total 
Project Independence expenditures (box 3C) per control group member, appear in Figure 4.1 as 
"dotted" boxes to indicate that these expenditures for controls were incurred contrary to the research 
design. When non-Project Independence expenditures (box 4C) are added to the Project Independence 
expenditures, the total gross cost per control group member (box 5C) is obtained, representing the 
government's total investment per control group member. 

The net cost of Project Independence, i.e., the cost per program group member over and above 
the cost per control group member (see Section VI), is represented in Figure 4.1 by box 6. The net 
cost is obtained by subtracting the total gross cost per control group member from the total gross cost 
per program group member. 



!!• Average Cost of Proiect Independence to HRS per Program Group Member 
(Figure 4 A. Box IP) 

HRS expenditures for Project Independence consisted of operating costs and support service 
payments. Operating costs largely reflected the costs of case management - e.g., following up on 



The cost analysis presented in this chapter uses an analytic approach similar to that used in MDRC's 
previous evaluations of welfare-to-work programs, particularly in the cost analysis for California's GAIN 
program (Riccio, Friedlander, and Freedman, 1994). Many of the techniques were originally developed for 
the evaluations of state programs in MDRC*s Demonstration of State Work/Welfare Initiatives; see Long and 
Knox, 1985, for additional information. 
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FIGURE 4.1 

THE MAIN COMPONENTS OF GROSS AND NET COSTS 




Program Group 



Project Independence-related 
expenditures by HRS for 
administration, case manage- 
ment, monitoring, employ- 
ment-related activities, 
and support services 
(including child care) 



Project Independence-related 
expenditures by non-HRS 
agencies (for Project 
Independence participants) 
for employment-related 
activities 




Control Group 



(a) 



Same as 1 P 



Non-Project independence 
cost per program group 
member: expenditures by 
non-HRS agencies for non- 
Project Independence (i.e., 
post-Project Independence) 
employment-related 
activities and child care 



Total Cost of 
Project Independence per 
Program Group Member 



Total Gross Cost 
per 

Program Group Member 



@(?) 

: Same as 2P 



(a) 



WW 



Non-Project Independence 
cost per control group 
member: expenditures by 
non-HRS agencies for non- 
Project Independence 
employment-related 
activities and child care 



Total Cost of 
Project Independence per 
Control Group Memtier 



Total Gross Cost 
per 

Control Group Member 



Difference = 
Net Cost per Program Group Member 

[©= @-@ ] 



NOTE: (a) As discussed in Chapter 3, contrary to the research design, some control group members 
received Project Independence services. The cost of these services was included in the total gross cost per 
control group member. 



-65- 



ERIC 



X 1-^ t 



applicants and recipients who failed to attend their scheduled orientation appointments, providing initial 
and subsequent assessments and employability planning, and referring and monitoring program group 
members assigned to independent job search, education, and training activities. HRS operating costs 
also included the costs of operating job clubs, providing other job search assistance, and implementing 
job development.-' The following section presents an estimate of HRS operating costs per program 
group member within two years after random assignment, and how these costs were divided across 
specific program functions. The chapter then turns to a discussion of HRS support service 
expenditures, which cover Project Independence child care and other support services to enable 
individuals to participate in program activities. 

A. Operating Costs (Figure Box IP) 

Project Independence operating costs covered expenditures for all program group members and, 
for this analysis, have been divided among eight main program activities or functions: orientation, 
formal assessment and employability planning, independent job search, job club, adult basic education 
(ABE) and GED preparation classes, English as a Second Language instruction, vocational training 
and post-secondary training, and on-the-job training.^ An overview of what these costs consisted of 
follows. 

1. Orientation . Expenditures allocated to this activity included the cost of staff time and 
overhead for working with both orientation attenders and those who were referred to the program but 
did not attend orientation (about 23 percent of those referred to Project Independence). For orientation 
attenders, staff time was spent explaining the participation mandate and the Project Independence 
activities in group sessions; conducting brief, individual, initial assessments to determine job-readiness; 
and assigning an individual to her first activity or granting (and monitoring) a deferral (i.e., temporary 
suspension of the participation requirement). For orientation "no-shows," staff time was spent 
rescheduling appointments and, for those who remained mandatory for Project Independence but failed 
to attend orientation, implementing the sanctioning process.^ 

2. Formal assessment and employability planning . Costs covered salary plus overhead 
expenditures for case managers' time spent on formal assessments (including literacy testing) and the 
development of an Employability Plan for participants who were judged to be not job-ready as well 
as for those who completed an up-front independent job search without finding a job. 

3. Independent job search and lob club « HRS expenditures for independent job search 
covered staff time and overhead costs for monitoring individuals assigned to independent job search, 
meeting with those who did not find a job to assign them to their next activity, and doing the 
paperwork for initiating the sanctioning process for noncompliant individuals. Similar expenses were 
incurred for persons assigned to job club activities. In addition, costs for job club covered the staff 
time required to conduct job club sessions (or the costs of paying others - JTPA, LES, or another 



•^Job development staff promoted Project Independence within the community, researched the local labor 
market, contacted employers, compiled lists of job opportunities, and posted job openings. 

^As noted in Chapter 3, Project Independence provided work experience activities in some counties. 
However, because this evaluation did not find evidence that the members of the research sample participated 
in these activities, such costs are not included in the analysis. 

^Some AFDC applicants who were referred to Project Independence were not approved for AFDC and 
thus were no longer mandatory for Project Independence. 
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provider - to run the clubs). In this analysis, expenditures for staff time spent on job development 
were also included in the cost of job search and job club activities. 

4. Basic education, vocational training and post-secondary education, and on-the-iob 
training . As discussed in Chapter 3, basic education encompassed aduh basic education (ABE), GED 
preparation, and instruction in English as a Second Language (ESL). For Project Independence 
participants who had a high school diploma or GED certificate, vocational training, post-secondary 
education, and on-the-job training were offered (if they were consistent with the participant's 
Employability Plan). The costs incurred by HRS for these activities were for Project Independence 
case managers' salaries (together with overhead expenditures) and not for the education or training 
itself, which was usually provided by adult schools, vocational training centers, and colleges (mainly 
community colleges). Project Independence staff counseled participants, made referrals to education 
and training institutions, followed up on no-shows and dropouts, and applied the sanctioning 
procedures for noncompliance with the participation mandate. Case managers also made referrals for 
child care services and arranged for transportation and other support services. (HRS child care and 
other support service payments to participants or providers were accounted for separately, as discussed 
in the next section.) 

HRS's average operating cost per program group member for a specified activity was usually 
calculated by first estimating a unit cost (i.e., the average operating cost per participant per month - 
or per "participant-month" — for the activity).^ The unit cost was then multiplied by the average 
length of time (in months) that people participated in the activity. 

To determine the unit cost, the total HRS expenditures for the activity for a specified period 
of time were divided by the total number of participant-months in that activity over the same period. 
The HRS expenditures were taken from the quarterly Project Independence expenditure reports of 
HRS's Economic Services Division for the period from July 1990 (the beginning of random 



^A person was counted as having participated in an activity in a given month if she participated in it at 
least one day that month. In the case of orientation and formal assessment, the unit cost was per session. 
rather than per participant-month. (Each individual's orientation and each formal assessment was counted as 
a single session, even if it extended beyond one day or took place in a group setting, as was common for 
orientation.) The unit cost was then multiplied by the total number of sessions of orientation or formal 
assessment to obtain the cost of each of these components. 

In order to avoid double counting of some education and training costs, the cost of HRS contracts for 
education and training (approximately 2 percent of total HRS expenditures for the period used for this report) 
was excluded in calculating HRS unit costs because the full cost of education and training was captured by 
the method used to calculate the costs incurred by the education and training providers themselves, as 
discussed in Section IIL 

Calculations of HRS units costs included all those who received Project Independence services - both 
the program group members who participated in Project Independence and the control group members who 
(contrary to the research design) participated. In other words, for each component, there is a single HRS unit 
cost per participant, which applies to control group participants as well as program group participants. These 
unit costs are presented in Table 4. 1. As was true for the program group, the duration of participation for 
control group members (i.e., the number of months or sessions in which they participated in Project 
Independence) was based on data from the 24-month survey of a subsample of program and control group 
members. See Appendix C for further details. 
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assignment) through June 19927 The number of participant-months in each activity was obtained 
from the on-line Project Independence Information System, in which case managers entered the dates 
of each registrant's activities. However, for the purposes of this report, the cost analysis used the four 
consecutive quarters from July 1991 through June 1992. This is because monthly participation counts 
for each activity could be obtained for this same 12-month period, but not for the prior year.^ All 
expenditures were inflation-adjusted to 1993 dollars before unit costs were estimated. 

Unit cost estimates for each of the main Project Independence activities are displayed in Table 
4.1. It is important to note that although unit costs apply to activities, they include the cost of working 
with nonparticipants. For example, if, in a given month, 20 clients are assigned to a job club but only 
15 participate, the cost of job club per participant in that month includes the direct and indirect costs 
of the staff time spent with five "no-shows," 

On the whole, these HRS unit costs were relatively low,^ For example, the estimated HRS 
unit cost for independent job search was only $72 per person per month, and it was not much higher 
for basic education activities (ABE, GED, and ESL). (However, as noted earlier, these costs did not 
include the costs incurred by the schools,) One factor keeping the unit costs low was the relatively 
high client-to-caseworker ratio in Project Independence, Indeed, program group members were 
referred to Project Independence, and participated in its activities, during a time when caseloads were 
increasing. According to the Staff Activities and Attitudes Survey, the average client-to-caseworker 
ratio was over 200 by October 1991 and may have continued to increase after that date. At the same 
time. Project Independence expenditures remained approximately the same,^^ Therefore, the HRS 
unit costs were probably lower during July 1991 through June 1992 (the period used for the estimates 
in this report) than they were in the prior year (a period in which a large share of the research sample 



^As noted in Chapter 2, the main HRS expenditure forms used were the ACF-332 (and the earlier report 
form, FSA-104), quarterly reports that Project Independence sends to the federal government listing 
expenditures by activity, (The Administration for Children and Families - ACF - was formerly called the 
Family Support Administration — FSA. It is the part of the U,S. Department of Health and Human Services 
responsible for the AFDC and JOBS programs,) For those reports, the allocation of costs to each activity is 
based on quarterly "random moment samplings*' (RMS) conducted by the state to determine the percentage 
of time staff devoted to each activity. In this analysis, the cost of staff time devoted to administrative and 
other non-activity-specific tasks was reallocated proportionately to the various activities. Finally* 
approximately 4 percent of the total expenditures were for education and training support services, such as 
books and tuition, paid directly to the provider or client. These costs were allocated directly to the specific 
activities for which they were used. Work-related support services, such as child care and transportation, 
were excluded from the operating costs. 

^Unit costs during this period may have been somewhat lower than in the previous year. The implications 
for the cost analysis are discussed below. 

^Unit costs were estimated to be much higher for California's GAIN program (see Riccio, Friedlander, 
and Freedman, 1994). For example, in GAIN, the county-specific unit costs for the combination of individual 
job search and group job club activities (including the cost of following up on nonparticipants) ranged from 
$537 to $1,120, depending on the county. 

^^As discussed in Chapter 1, AFDC caseloads in Florida were increasing during the first two years of 
follow-up, resulting in an increase in the number of participants served by Project Independence, while 
expendimres for Project Independence remained fixed (or decreased slightly). 
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TABLE 4.1 



ESTIMATED UNIT COSTS FOR EMPLOYMENT-RELATED ACTIVITIES (IN 1993 DOLLARS) 



HRS Unit Cost 
(Marked Up to Include Costs 
for F ollowing Nonparticlpan ts) _ 



Component 
Orientation 

Formal assessment/ employabillty planning 

Independent job search 

Job club 

ABE or GEO 

ESL 

Vocational training or post- 

secondary education 
On-the-job training 
Unpaid work experience 



Non-HRSUnit Cost 



Average per 
Participant- 
Month ($) 
(A) 


Average per 
Session ($) 
. . _ (B) 


Average per 
Participant- 
Month ($) 
(C) 


Average 
per FTE (a) ($) 
- . (D). 


n/a 


69 


n/a 


n/a 


n/a 


112 


n/a 


n/a 


72 


n/a 


72 


n/a 


226 


n/a 


226 


n/a 


76 


n/a 


n/a 


2688 


76 


n/a 


n/a 


2688 


21 


n/a 


n/a 


4768 (b) 


231 


n/a 


231 


n/a 


30 


n/a 


30 


n/a 



SOURCES: MDRC calculations based on fiscal and participation data from the Florida Department of Health and Rehabilitative Services (HRS), 
the Florida Department of Education, and information from the 24 -month survey of a subsample of program and control group members. 

NOTES: Where data are not applicable, "n/a" is used. 

(a) One FTE (i.e., one 'lull -time equivalent" student) represents the total number of scheduled hours for a student attending full-time 
for one academic year. 

(b) The average cost per FTE is a weighted average of the costs per FTE of vocational training centers and community colleges. The 
unit cost for the program group was $4,768 per FTE. The unit cost for the control group was $4,758 per FTE because control group members had 
a slightly different proportion of participants attending these institutions. 



was served). The HRS unit costs presented here should be viewed as lower-bound estimates. ^ 
Some evidence, based on estimated annual counts of the number of participants in Project 
Independence^^ in each year (July 1990 through June 1991 and July 1991 through June 1992), 
suggests that HRS unit costs may have been up to 20 percent higher in the first year. As previously 
noted, the HRS average operating cost of a Project Independence activity per program group member 
was obtained by multiplying the average unit cost by the corresponding average length of stay. Table 
4.2 presents the results for a period of two years after random assignment. As colimin A of the top 
panel shows (on the subtotal line), the estimated two-year average operating cost was $312 per person 
in the program group. If, however, it is assumed that HRS unit costs were 20 percent higher the 
previous year (as discussed above), the effect on the HRS average operating cost would be an increase 
of only $62, from $312 to $374. 

B. Child Care and Other Support Service Costs (Figure 4.1. Box IP) 

The total cost of Project Independence included expenditures for support services to enable 
those referred to the program to participate in it. These services were child care, transportation, and 
miscellaneous expenses (e.g., for tools, uniforms, and equipment). Project Independence participants 
who engaged in an education or training activity and needed assistance with child care were referred 
to the Central Child Care agencies^-^ in their local community. In contrast, child care for participants 
in short-duration components (e.g., independent job search) was arranged by Project Independence 
staff. All Project Independence child care is paid for through Family Support Act (FSA) child care 
funds (Title IV-A). However, access to these child care resources was severely curtailed in January 
1991, when the demand for them exceeded the amount of program dollars allocated for child care.^^ 



^^As previously noted, random assignment took place from July 1990 through August 1991. Because the 
period used for estimating unit costs was July 1991 through Jime 1992, program group members entering 
Project Independence toward the end of the random assignment period and those who had longer lengths of 
stay would have been eligible to participate in activities during that period. It is estimated that one-third of 
all participation during the first two years took place during the July 1991 through June 1992 period, and that 
two-thirds of the participation occurred during the prior year. 

^^The data used to estimate annual counts are from the 1990-91 and 1991-92 Project Independence Annual 
Management Report prepared by HRS and LES, respectively. 

^^In Florida, the Children, Youth, and Families (CYF) division of HRS contracts with Central Child Care 
agencies in the counties to provide referrals for child care. The Central Child Care agencies are the conduits 
for paying providers for Project Independence-subsidized child care as well as other types of publicly funded 
child care. 

^"^Case managers at the local Project Independence offices authorized and arranged payment for up to 30 
days of child care. In October 1992, it became mandatory for all child care to be arranged and paid for 
through the Central Child Care agencies. 

^^FSA (IV-A) funds are also used to pay for child care for working AFDC recipients and transitional child 
care for people who leave AFDC for employment. Both such uses are entitlements. As AFDC caseloads and, 
consequently, the number of Project Independence participants increased, the demand for child care also grew, 
but without a proportionate increase in the amount of child care dollars available. Restrictions on the 
availability of Project Independence child care were implemented at different rates and at different levels, 
depending on the HRS service district (and, therefore, county). However, all districts began implementing 
cutbacks in January 1991, and the cutbacks expanded thereafter, with Project Independence child care being 

(continued...) 
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TABLE 4.2 



ESTIMATED PROJECT INDEPENDENCE COST PER PROGRAM AND 
CONTROL GROUP MEMBER WITHIN TWO YEARS AFTER RANDOM ASSIGNMENT. 
BY SERVICE COMPONENT AND AGENCY (IN 1993 DOLLARS) 



Program Group Control Group (a) 





Average per 


Percentage 


Average per 


Percentage 




Person ($) 


Distribution 


Person ($) 


Distribution 


Agency and Component 




_(A) 


(B) 


(C) 


(D) 


HRS Project Indopendence Cost 












Orientation 




67 


15.6 


13 


12.4 


Pnrm^if a<5QP^Qmpnt/ pmnlovahilitv nianninn 




55 


1 2.8 


7 


6.7 


Independent job search 




58 


13.5 


10 


9.5 


Job club 




43 


10.0 


7 


6.7 


ABE or GEO 




25 


5.8 


3 


2.9 


ESL 




1 2 




4 

1 


1 .U 


Vocational training or post-secondary education 




17 






1 Q 

1 .y 


On-the-job training 




35 


A 1 
O. 1 


A 
** 


O.O 


Unpaid work experience 




0 


0.0 


0 


0.0 


Subtotal (operating costs) 




312 


72.6 


47 


A A Q 

44. o 


Child care 




80 


18.6 


52 


49.5 


Other support sers/ices 




38 


8.8 


6 


5.7 


Total 




430 


100.0 


105 


100.0 


Non-HRS Agencies' Project Independence Cost (b) 










Orientation 




0 


0.0 


u 


U.U 


Fornnal assessment/ employability planning 




0 


0.0 


0 


0.0 


Independent job search 




0 


0.0 


0 


0.0 


Job club 




0 


0.0 


0 


0.0 


ABE or GED 




65 


13.2 


8 


14.0 


ESL 




30 


6.1 


3 


5.3 


Vocational training or post -secondary education 




396 


80.7 


46 


80.7 


On-the-job training 




0 


0.0 


0 


0.0 


Unpaid work experience 




0 


0.0 


0 


0.0 


Subtotal (operating costs) 




491 


100.0 


57 


100.0 


Total Project Independence Cost 












Orientation 




67 


7.3 


13 


8.0 


Formal assessment/ employability planning 




55 


R n 

D.U 




4 


Independent job search 




58 


6.3 


10 


6.2 


Job club 




43 


4.7 


7 


4.3 


ABE or GED 




90 


9.8 


11 


6.8 


ESL 




42 


4.6 


4 


2.5 


Vocatranal training or post-secondary education 




413 


44.8 


48 


29.6 


On-the-job training 




35 


3.8 


4 


2.5 


Unpaid work experience 




0 


0.0 


0 


0.0 


Subtotal (operating costs) 




803 


87.2 


104 


64.2 


Child care 




80 


8.7 


52 


32.1 


Other support sers/ices 




33 


4.1 


6 


3.7 


Total 




921 


100.0 


162 


100.0 



SOURCES: See Table 4.1 . Child care data are from Florida's Central Child Care agencies and the l2-monthsun/ey of a 
subsample of program group members. 



NOTES. Rounding may cause slight discrepancies in calculating .>ums. averages, and differences. 

(a) As discussed in Chapter 3. contrary to the research d^^sign. some control group members received Project 
Independence services. The cost of these services was included in th.e Project Independence cost per control group 
member. 

(b) Non-HRS agencies did not pay for any Project Independence support services. 
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HRS restricted access to Project Independence child care to participants who were already taking part 
in an activity and receiving child care, although a small amount of child care money remained 
available for those who were newly entering independent job search or job club. The average child 
care costs presented here cover two years of follow-up, which span the period of change in child care 
availability. 

To estimate Project Independence child care costs, information on monthly payment amounts 
and the number of months that these payments were made was collected from the Central Child Care 
agencies' records for a subsample of program group members. For those who participated in an 
education or training activity and received child care, the average child care payment was $275 a 
month and reached a total of $1 ,438 per person using child care within the first two years after random 
assignment. Participants in independent job search or job club (and other short-term components) who 
used child care did so for an estimated one month, implying a total cost of $275. Overall, however, 
according to the evaluation's 12-month survey of a subsample of program group members, only 11 
percent of orientation attenders reported using any Project Independence child care.^^ Thus, when 
the cost of all Project Independence child care was averaged over all program group members 
(including those who did not use it), the two-year cost was estimated to be only $80 per person (see 
Table 4.2, column A in the top panel). 

Project Independence participants were also eligible for support service payments for 
transportation and miscellaneous expenses for work-related needs. The two-year cost of these other 
services is estimated to have been $38 per program group member (see Table 4.2, column A of the 
top panel). 
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^^(... continued) 

virtually eliminated in many districts, although some child care remained available for job search activities 
lasting less than one month. 

^^Individual-level child care data were collected for up to 42 months from the Central Child Care agencies 
in four counties: Broward, Dade, Duval, and Orange. The sample in each county was 150 program group 
members and 50 control group members who were randomly assigned in September and October 1990 (in 
Broward, Dade, and Orange) and October and November 1990 (in Duval). The duration of child care use 
was calculated separately for the program and control groups in all four counties combined. The average child 
care payment was calculated as the average for the program and control groups combined for Duval and 
Orange counties, the two of the four counties in which the records showed actual payment amounts. 

^^The subsample included individuals randomly assigned both before and after the child care restrictions 
went into effect, so the 11 percent rate represents child caie use for the full sample. 

^^The child care cost of $80 per program group member was calculated by weighting the child care cost 
per person in education or training and the child care cost per person in independent job search or job club 
by the rates of participation in these activities and then applying the average cost per person using child care 
to all program group members. 

^^The cost of other support services per program group member was estimated to have been 12.2 percent 
- or $38 - of the $312 HRS operating cost per program group member, based on HRS expenditures from 
July 1991 through June 1992 for everyone in Project Independence. Some additional support service costs 
for books, tuition, etc., were paid directly to clients for education and training activities and were included 
in the total operating cost per program group member. 
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III. Average Cost of Project Independence to Non-HRS Agencies per Program Group Member 
(Figure 4,1. Box 2P) 



Non-HRS agencies provided education and training for program group members. Although 
HRS did not directly incur the costs of these activities, those expenditures were counted as Project 
Independence-related costs. This is because they covered activities that program group members were 
referred to (or continued in, if their participation was self-initiated) in accordance with the Project 
Independence participation mandate. 

To estimate these non-HRS costs, the types of instimtions used by the program group for 
education and training had to be determined and data on their unit costs had to be obtained. The 
evaluation's 24-month survey showed that program group members participated in basic education 
(ABE, GED, and ESL) activities primarily at adult schools. In addition, it showed that for vocational 
training and post-secondary education, the main providers were vocational training centers and 
conmiunity colleges. The unit cost for each institution is expressed as the cost per PTE (i.e., one 
"full-time equivalent" student). One PTE unit represents the total number of scheduled hours for 
a student attending full-time for one academic year.^^ Using this information, a unit cost 
representing all non-HRS agencies providing vocational training or post-secondary education was 
calculated. 

The unit costs appear in colunui D of Table 4. 1 . Non-HRS agencies spent an estimated $2,688 
per PTE for ABE, GED, and ESL activities and almost twice as much ($4,768 per PTE) for vocational 
training and post-secondary education. 

Multiplying the unit cost for each activity by the corresponding average number of PTEs per 
program group member yields the average cost of each activity for the program group. Por 



^"The adult school cost data are from the Florida Department of Education, Division of Public Schools, 
Bureau of School Business Services, Financial Management Section. The reports used were: Profiles of 
Florida School Districts: 1990-91 Financial Data Statistical Report md 1990-91 Program Cost Analysis Report 
Series, Numbers 1 and 5. These reports also provided cost data regarding the vocational training centers. 
Community college cost data are from the Florida Department of Education, Division of Community Colleges. 
The reports used were: 1990-91 Cost Analysis: CA-2 Systemwide Summary and Summary of Revenue, 
Expenditures, and Fund Balances by General Ledger Code for the Fiscal Year 1990-91. 

^^For adult schools, vocational training centers, and non-credit courses at community colleges, one PTE 
equals 900 scheduled hours of class, which is equivalent to 180 days of five one-hour classes per day, or 36 
weeks with 25 scheduled hours of class per week. Hours for a student attending less than a full schedule 
equal a fractional FTE. For community college credit courses, one FTE equals 30 credit hours, which MDRC 
estimated to be equivalent to 15 credits for each of two semesters, where a semester is 18 weeks and the class 
meets one hour each week for each credit taken. Therefore, one community college credit FTE is equivalent 
to 540 scheduled hours (15 credit hours times 18 weeks times 2 semesters equals 540 hours). 

^^A unit cost per FTE for vocational training and post-secondary education was calculated by weighting 
the vocational training unit cost and the community college unit cost by the percentage of the program group 
attending a vocational training or post-secondary activity at each institution. 

^^The average number of PTEs was calculated by mukiplying the average length of stay in months by the 
number of scheduled hours per month, and then dividing by the number of scheduled hours for one FTE (as 
defined in footnote 21). 
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example, the non-HRS Project Independence cost of ABE and GED was $65 per person in the program 
group (i.e., the unit cost of $2,688 per PTE multiplied by the average number of PTEs). The total 
non-HRS operating cost per program group member for all Project Independence-related activities was 
$491 (Table 4.2, column A of the middle panel). Of that amount, nearly 81 percent was spent on 
vocational training and post-secondary education. 



TV. Total Cost of Project Independence per Program Group Member (Figure 4,1, Box 3P) 

As illustrated in Figure 4.1, the total cost of Project Independence per program group member 
was the sum of the average costs incurred by HRS and non-HRS agencies. This was estimated to be 
$921 over the two-year period (Table 4.2, colunm A of the bottom panel) and was almost equally split 
between HRS ($430) and non-HRS agencies ($491). The percentage distribution in column B of the 
bottom panel shows that, of this total, 87 percent was accounted for by operating costs ($803), another 
9 percent was accounted for by child care expenditures ($80), and another 4 percent by support 
services ($38). The distribution of Project Independence costs for activities and su^l^rt services 
shown in Table 4,2 is graphically represented in Figure 4.2. 



V, Total Gross Cost (for Project Independence and Non-Proiect Independence Services) per 
Program Group Member (Figure 4,1. Box 5P) 

In addition to using Project Independence services, some program group members participated 
in vocational training and post-secondary activities on their own after their departure from Project 
Independence. Because these services have the potential to increase program group members' longer- 
term earnings and reduce their use of welfare (effects that would be captured as "benefits" in the 
benefit-cost analysis, which is presented in Chapter 7), it is important to count their cost in estimating 
the total government investment for program group members. This total investment is the total gross 
cost per program group member and is the amount that must be compared to the total gross cost per 
control group member in order to determine the net cost per program group member. 

A* Non-Proiect Independence Operating Cost per Program Group Member (Figure 4,1, 
Box 4P) 

The estimation of non-Project Independence (i.e., post-Project Independence) operating costs 
employed the same method that was used to calculate non-HRS provider costs (see Section III): The 
provider unit cost was multiplied by the program group members' average length of stay in the 
specified activity. The results for the two-year period are presented in cclumn A of the middle panel 
of Table 4.3. Column B displays the percentage distribution. It is noteworthy that vocational training 
and post-secondary education accounted for 80 percent of the non-Project Independence two-year cost 
of $1,041 per program group member. 

B. Non>Project Independence Child Care (Figure 4.1. Box 4P) 

Program group members could receive other types of child care (not paid for by Project 
Independence) after leaving Project Independence if they met the eligibility requirements. "AFDC- 
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FIGURE 4.2 



PERCENTAGE DISTRIBUTION OF THE ESTIMATED TWO-YEAR TOTAL COST 
OF PROJECT INDEPENDENCE FOR THE PROGRAM GROUP, BY SERVICE COMPONENT 




SOURCE: Table 4.2. 

NOTES: Distribution may not sum to 100.0 percent because of rounding. 

The estimated two-year total cost of Project Independence per program group 
member was $921 and included HRS and non-HRS expenditures. 
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TABLE 4.3 



ESTIMATED PROJECT INDEPENDENCE. NON-PROJECT INDEPENDENCE. AND TOTAL GROSS COST 
PER PROGRAM AND CONTROL GROUP MEMBER WITHIN TWO YEARS 
AFTER RANDOM ASSIGNMENT. BY SERVICE COMPONENT (IN 1993 DOLLARS) 



Program Group 



Component 



Average per 
Person ($) 

(AL 



Total Project independence Cost 
Orientation 

Formal assessment/ employability planning 

Independent job search 

Job club 

ABE or GEO 

ESL 

Vocational training or post-secondary education 
On-the-job training 
Unpaid work experience 
Subtotal (operating costs) 

Child care 

Other support services 
Total 



Percentage 
Distribution 
(BL 



Control Group (a) 



Average per 
Person ($) 

(CL 



Percentage 
Distribution 

(DL 



67 


7.3 


13 


8.0 


55 


6.0 


7 


4,3 


58 


6.3 


10 


6.2 


43 


4,7 


7 


4,3 


90 


9,8 


11 


6,8 


42 


4,6 


. 4 


2,5 


413 


44,8 


48 


29,6 


35 


3,8 


4 


2,5 


0 


0,0 


0 


0.0 


803 


87,2 


104 


64,2 


80 


8,7 


52 


32.1 


38 


4,1 


6 


3,7 


921 


100,0 


162 


100.0 



Non-Project Independence Cost 

Orientation 

Formal assessment/ employability planning 

Independent job search 

Job club 

ABE or GED 

ESL 

Vocational training or post-secondary education 
On-the-job training 
Unpaid work experience 
Subtotal (operating costs) 

Child care (b) 
Other support services (c) 
Total 



0 


0.0 


0 


0.0 


0 


0,0 


0 


0.0 


6 


0,6 


15 


1,6 


8 


0,8 


21 


2,3 


76 


7,3 


85 


9,3 


22 


2,1 


61 


6,7 


832 


79,9 


661 


72.5 


22 


2,1 


36 


3,9 


0 


0,0 


0 


0.0 


966 


92,8 


879 


96.4 


75 


7,2 


33 


3.6 


n/a 


n/a 


n/a 


n/a 


1041 


100,0 


912 


100,0 



Total Gross Cost 
Orientation 

Formal assessment/ employability planning 

Independent job search 

Job club 

ABE or GED 

ESL 

Vocational training or post -secondary education 
On-the-job training 
Unpaid work experience 
Subtotal (operating costs) 

Child care 

Other support services 
Total 



67 


3.4 


13 


1,2 


55 


2.8 


7 


0,7 


64 


3.3 


25 


2,3 


51 


2,6 


28 


2,6 


166 


8,5 


96 


8,9 


64 


3,3 


65 


6.1 


1245 


63,5 


709 


66,0 


57 


2,9 


40 


3,7 


0 


0,0 


0 


0,0 


1769 


90,2 


983 


91,5 


155 


7,9 


85 


7,9 


38 


1,9 


6 


0.6 


1962 


100,0 


1074 


100,0 



SOURCES: See Table 4,1 , Child care data are from Florida's Central Child Care agencies and the 1 2 -month survey of 
a subsample of program group members. 

NOTES: Rounding may cause slight discrepancies in calculating sums, averages, and differences, 

(a) /Vs discussed in Chapter 3, contrary to the research design, some control group members received Project 
Independence services. The cost of these services v/as included in the Project Independence cost and total gross cost 
per control group member, 

(b) Child care costs include three types of non -Project Independence child care: AFDC-employed child care, 
transitional child care, and income-eligible (working poor) child care, 

(c) Costs are not available (n/a) because data were not collected on the use of other support services. Although 
both program and control group members who participated in non -Project Independence activities may have received some 
oublic subsidies, the cost was probably low because many education and training providers do not offer other support services. 
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employed" child care is available in Florida to welfare recipients who are also employed. 
(Usually, but not always, a program group member who was receiving AFDC-employed child care 
had been deregistered from Project Independence.) Also, under federal regulations, welfare recipients 
who leave AFDC for employment are entitled to receive 12 months of "transitional" child care. 
Afterwards, sample members whose income is below a prescribed level are eligible for "income- 
eligible" child care, which is also called "working poor" child care. 

Since being referred to Project Independence is not a condition for receiving these other types 
of child care, the payments were not considered a Project Independence-related cost in this analysis. 
However, Project Independence may have increased the use of these types of child care if it succeeded 
in increasing the percentage of the program group who left AFDC for employment and to the extent 
that Project Independence staff n^iade recipients aware of their eligibility for other types of child care. 
Thus, the cost of these other types of child care were considered a "complementary" cost of Project 
Independence rather than a direct cost. For these reasons, it is appropriate to discuss them in an 
analysis of the total cost of serving the program gi'oup (and in the benefit-cost analysis). Therefore, 
this cost — estimated to be $75 per program group member over two years^^ — is listed in Table 
4.3 (column A in the middle panel) as the non-Project Independence cost of child care.^^ 

C. Total Gross Cost per Program Group Member (Figure 4.1^ Box 5P) 

The two-year total gross cost per program group member is obtained by adding the top-panel 
estimates of Project Independence costs in column A of Table 4.3 to the middle-panel estimates of non- 
Project Independence costs. As shown in colunm A of the bottom panel, the estimated total gross cost 
per program group member was $1,962. Colunm B of the bottom panel indicates that 64 percent of 
the total gross cost was spent on vocational training and post-secondary education. When ABE and 
GED costs (8.5 percent) and ESL costs (3.3 percent) are added in, all education and training activities 
combined accounted for three-fourths of the average total gross cost of Project Independence for the 
program group. Child care accounted for 7.9 percent of the total. 



VL Total Gross Cost per Control Group Member (Figure 4.1, Boxes 1C-5C) 

Ideally, the total gross cost per control group member would indicate what the government 
would have spent on employment-related services for the program group in the absence of Project 
Independence. (Some members of the program group, on their own initiative, would have participated 
in activities available in the community.) In this evaluation, the expenditures for control group 



^In Florida, some AFDC recipients who are employed receive AFDC-employed child care in lieu of an 
AFDC child care disregard. 

^^The breakdown was as follows: AFDC-employed child care, $25; transitional child care, $22; and 
working poor child care, $28. 

^^Individual-level payment data were collected for the same sample as for Project Independence child care 
(see the description of that sample in footoote 16). The rate of child care usage, the average monthly payment 
for those receiving child care, and the average number of months (length of stay) were calculated separately 
for each type of child care. The same methodology used for calculating the cost of Project Independence child 
care was used to calculate these other types of child care per program group member. 
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members were primarily for services they used on their own initiative, without the assistance of Project 
Independence staff. However, contrary to the research design, an estimated 20 percent of control 
group members received some aspect of the Project Independence treatment. (See Chapter 3 and 
Appendix C for a full discussion of this issue.) 

A. Project Independence (HRS and non-HRS) Cost per Control Group Member (Figure 
4.1, Box 3C) 

Table 4.2 (colunm C of the bottom panel) shows that the estimated Project Independence 
operating cost per control group member was $104,^^ which is only 13 percent of the average 
operating cost for the program group. This is consistent with the estimate, presented in Chapter 3, 
that 20 percent of the controls were exposed to Project Independence, but that only 7.5 percent 
participated in its employment-related activities. This implies that controls who were exposed to 
Project Independence received a less intensive set of services than did program group members, 
perhaps because some of them were identified at the initial orientation and thereafter were no longer 
served by Project Independence. 

Under the research design, control group members who participated in a non-Project 
Independence education or training activity on their own initiative were eligible to receive Project 
Independence-funded child care on the same basis as members of the program group. The cost of that 
child care is estimated to have been $52 per control group member over the two years following 
random assignment (see Table 4.2), or almost two-thirds the $80 cost per program group member. 
Control group members were not eligible for other support services. However, because some of them 
entered Project Independence, the program incurred a two-year average cost of $6 per control group 
member for those services (Table 4.3, column C of the top panel). Adding the costs of child care 
and other support services to the operating cost produces a total cost of Project Independence per 
control group member of $162 over the two-year period. 

B. Non-Project Independence Cost per Control Group Member (Figuure 4.1, Box 4C) 

Control group members participated mostly in employment-related services that were not paid 
for or arranged by Project Independence. These activities accounted for approximately 85 percent of 
the total gross cost per control group member. Vocational training and post-secondary education 



" As noted earlier, there is a single HRS unit cost for each component, which applies to both control 
group members who received Project Independence services and program group participants. 

^^Based on information collected from Central Child Care agencies (as discussed in foomote 16), control 
group use of such child care for education and training activities (and, in a few cases, for Project 
Independence job search activities) was estimated to have been approximately 65 percent of program group 
child care use. Therefore, the cost of Project Independence child care per control group member was 
estimated to have been $52 (65 percent of $80). 

^^This estimate was based on the ratio of control group (7.5 percent) to program group (44.4 percent) 
participation in any Project Independence employment-related activity. Since 7.5 percent divided by 44.4 
percent is 17 percent, the control group cost of other support services was probably about 17 percent that of 
the program group cost, or $6. 

•^^No data were collected on the receipt of support services (other than child care) by members of the 
control group in the course of their participation in employment-related activities. (Similarly, no data were 

(continued...) 
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were by far the most widely used activities, accounting for 73 percent (Table 4.3, column D of the 
middle panel) of the two-year $912 cost per control group member for non-Project Independence 
services. Basic education (ABE, GED, and ESL) accounted for 16 percent of the non-Project 
Independence cost per control group member. Control group members made modest use of 
independent job search and job club activities, which together represented only 4 percent of the 
operating cost for non-Project Independence agencies. 

Data were also collected on control group members' use of non-Project Independence child 
care that is available to employed welfare recipients - i.e., AFDC-employed, transitional, and 
income-eligible (working poor) child care, as discussed in Section VB.*^^ The total two-year cost of 
that child care was estimated to have been $33 per control group member (Table 4.3, colunm C of the 
middle panel). 

C. Total Gross Cost per Control Group Member (Figure 4.1, Box 5C) 

The total gross cost per control group member is the average cost of all non-Project 
Independence and Project Independence services used by control group members. As shown in Table 
4.3 (colunm C of the bottom panel), this was estimated to have been $1,074 per control group member 
for two years. 



VIL Net Cost per Program Group Member (Figure 4.1, Box 6) 

The net cost is determined by subtracting the total gross cost per control group member from 
the total gross cost per program group member (column B of Table 4.4 subtracted from column A). 
As presented in column C, the estimated net cost per program group member was $888 for two years. 
Column C also shows the net cost of each activity. The two-year net cost of vocational training and 
post-secondary education, $536, accounted for over half of the total net cost. 



VIIL Five- Year Cost Estimates 

The two-year cost estimates do not capture the entire cost of Project Independence and non- 
Project Independence employment-related activities. The 24-month survey found that 21 percent of 
the program group and 15 percent of the control group were still participating in activities at the end 
of the 24-month follow-up period, implying that costs were still being incurred beyond that point. 



^^(.., continued) 

collected on the use of such services by program group members while participating in non-Project 
Independence education and training activities.) 

^^Individual-level payment data were collected for the same sample that was used for collecting 
information on Project Independence child care (see footnote 16). The rate of child care use, average monthly 
payment for those receiving child care, and average number of months (length of stay) were calculated 
separately for each type of child care. The same methodology used for estimating the cost of Project 
Independence child care per control group member was used for estimating the cost of each type of child care 
per control group member. 

^^The breakdown is as follows: AFDC-employed child care, $21; transitional child care, $5; and income- 
eligible (working poor) child care, $7. 
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TABLE 4.4 



ESTIMATED TOTAL GROSS COST PER PROGRAM AND GON7BOL GROUP MEMBER 
AND NET COST PER PROGRAM GROUP MEMBER. WITHIN TWO YEARS AFTER RANDOM ASSIGNMENT. 

BY SERVICE COMPONENT (IN 1993 DOLLARS) 



Component 


Total Gross Cost 
per Program Group 
Member ($) 
(A) 


Total Gross Cost 
per Control Group 
Member (a) ($) 
(B) 


Net Cost per 
Program Group 
Member (A-B) ($) 
(C) 


Orientation 


67 


13 


54 


Formal assessment/ employability planning 


55 


7 


48 


Independent job search 


64 


25 


39 


Job club 


51 


28 


23 


ABE or GEO 


166 


96 


70 


ESL 


64 


65 


-1 


Vocational training or post --secondary education 


1245 


709 


536 


On-the-job training 


57 


40 


17 


Unpaid work experience 


0 


0 


0 


Subtotal (operating costs) 


1769 


983 


786 


Child care (b) 


155 


85 


70 


Other support services (c) 


38 


6 


32 


Subtotal (support services) 


. 193 


91 


102 


Total 


1962 


1074 


888 



SOURCES: See Table 4.1 . Child care data are froin Florida's Central Child Care agencies and the 1 2-month survey of 
a subsample of program group members. 

NOTES: Rounding may cause slight discrepancies in calculating sums, averages, and differences. 

(a) As discussed in Chapter 3, contrary to the research design, some control group members received Project 
Independence services. The cost of these services was included in the total gross cost per control group member. 

(b) Child care costs include Project Independence child care and three types of non-Project Independence 
child care: AFDC-employed child care, transitional child care, and income-eligible (working poor) child care. 

(c) The other support services cost is for Project Independence other support services. Data on the cost of 
other support services for non -Project Independence activities were not collected. 
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Thus, just relying on the two-year costs would underestimate the full cost of Project Independence and 
non-Project Independence activities. Costs incurred after the second year are important to consider 
because the additional investment may contribute to whatever post-two-year earnings gains or welfare 
savings (and other effects) are captured on the "benefits" side in the comparison of benefits and costs 
(see Chapter 7). Therefore, costs have been projected for years 3 to 5, based on trends in participation 
patterns from the first year of follow-up to the second. The projected costs of years 3 to 5 were 
added to the two-year costs (discussed in Sections II to VI) to obtain the estimated five-year costs. 

The five-year costs must be viewed with caution because they include three years of projected 
costs covering an unstable period in the life of Project Independence. During the first two years of 
follow-up, AFDC caseloads and Project Independence caseloads were rising, but resources remained 
the same or were slightly shrinking (see Chapter 1). Therefore, there was increased strain on the use 
of Project Independence resources, affecting, over time, sample members' access to program activities 
and support services (e.g., child care) and the intensity of case management. The changing 
environment thus makes it more difficult to predict sample members' future use of program resources. 

A, Five- Year Total Gross Cost per Program Group Member 

The estimated total gross cost per program group member for five years is $3,096 (Table 4.5, 
last row of column C). This reflects the addition of $1,134 in projected expenditures for years 3 to 
5 to the two-year gross cost per program group member. Most of this increase is for continued 
participation in vocational training and post-secondary education (not shown in the table). The 
estimated five-year Project Independence-related cost is $1,304 of the average total cost for the 
program group; the distribution across program components is depicted in Figure 4.3. 

The estimated five-year total gross cost per program group member is a little less than two- 
thirds of the $4,895 estimated five-year cost of California's Greater Avenues for Independence (GAIN) 
Program. This is consistent with Project Independence's up-front emphasis on independent job search 
and GAIN'S more costly emphasis on up-front basic education along with its inclusion of other 
education and training activities and largely group-based job search activities. In addition. Project 
Independence had a client-to-staff ratio roughly double that of GAIN. 

B. Five- Year Total Gross Cost per Control Group Member 

For control group members, the estimated five-year total gross cost is $1,946 (Table 4.5, 
column F), which is an additional $872 over the two-year gross cost of $1,074. As was true for the 
program group, control group members' continued participation in vocational training and post- 
secondary education accounts for most of the projected cost. 



^^The cost projection for years 3 to 5 used the same unit costs that were used for estimating the costs for 
the first two years. The unit costs were multiplied by a projected length of stay in each activity for years 1 
to 5. The projected length of stay in each activity took into account the participation "decay" rate from year 
I to year 2, the percentage who were still participating in the activity at the end of year 2, and an estimated 
percentage of those who began the activity for the first time during the projection period. See Appendix D 
for the average length of stay in each activity for years 1 to 5. 
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TABLE 4.5 



ESTIMATED TOTAL GROSS COST PER PROGRAM AND CONTROL GROUP MEMBER 
WITHIN TWO AND FIVE YEARS AFTER RANDOM ASSIGNMENT (IN 1993 DOLLARS) 



Average. Cost per Program Group Member Average Cost per Control Group Member (a) 



Cost Category 



Years 1 -2 
(Observed) ($) 

(AL 



Years 3-5 
(Projected) ($) 
[B) 



Years 1 -5 
(A+B) ($) 
(CL 



Years 1 -2 
(Observed) ($) 

(DL 



Years 3-5 

(Projc> ied) ($) 

(iL 



Years 1 -5 
(D+E) ($) 



Operating cost 
Project independence 
Non -Project Irxlependence 
Total 

Support services cost 
Project Independence (b) 
Non-Project Independence (c) 
Total 

Total gross cost 
Project Independence 
Non-Project Independence 
Total 



803 
966 
1769 



118 
75 
193 



921 
1041 
1962 



349 
712 
1061 



34 
39 
73 



383 
751 
1134 



1152 
1678 
2830 



152 
114 
266 



1304 
1792 
3096 



104 
879 
983 



58 
33 
91 



162 
912 
1074 



77 
769 
846 



16 
10 
26 



93 
779 
872 



181 
1648 
1829 



74 
43 
117 



255 
1691 
1946 



SOURCES: See Table 4.1 . Child care data are from Florida's Central Child Care agencies and the 1 2-month survey of a 
subsample of program group members. 

NOTES: Rounding may cause slight discrepancies in calculating sums, averages, and differences. 

(a) As discussed in Chapter 3, contrary to the research design, some control group members received Project 
Independence services. The cost of these services was included in the total gross cost per control group member. 

(b) The support services cost for Project Independence includes child care and other support services. 

(c) The support services cost for non-Project Independence is only for child care. Data were not collected on 
other support services. 



TABLE 4.6 

ESTIMATED TOTAL GROSS COST PER PROGRAM AND CONTROL GROUP MEMBER. 

AND NET COST PER PROGRAM GROUP MEMBER. 
WnHIN TWO AND FIVE YEARS AFTER RANDOM ASSIGNMENT (IN 1993 DOLLARS) 



Total Gross Cost Total Gross Cost Net Cost per 

per Program Group per Control Group Program Group 

Member ($) Member (a) ($) Member (A-B) ($) 

Cost Time Period (A) (B) (C) 

Two-year estimate (b) 

(observed) 1962 1074 888 
Five-year estimate (c) 

(observed plus projected) 3096 1946 1150 



SOURCES: See Table 4.1 . Child care data are from Florida's Central Child Care agencies and the 12-month survey of a 
subsample of program group members. 

NOTES: Rounding may cause slight discrepancies in calculating sums, averages, and differences. 

(a) As discussed in Chapter 3, contrary to the research design, some control group members received Project 
Independence services. The cost of these services was included in the total gross cost per control group member. 

(b) The total gross cost estimates per program and control group member are from Table 4.5, columns A and D, 

respectively. 

(c) The total gross cost estimates per program and control group member are from Table 4.5, columns C and F. 

respectively. 



FIGURE 4.3 



PERCENTAGE DISTRIBUTION OF THE ESTIMATED FIVE-YEAR TOTAL COST 
OF PROJECT INDEPENDENCE FOR THE PROGRAM GROUP, BY SERVICE COMPONENT 



Formal Assessment/ 
Employabilrty Planning 
5.3% 



Independent 
Job Search 
6.7% 



Other Support Services 

3.8% 
Orientatton i 

5.4% ^ Child Care 

7.9% 



Job Club 
4,0% 



ABE or GED 
8.0% 




Unpaid 
Work Experience 
0.0% 



On-the-Job Training 
3.9% 



Vocational Training or 
Post-Secondary 
Education 
51.8% 



SOURCES: Table 4.5 and MDRC five-year cost estimates. 

NOTES: Distribution may not sum to 100.0 percent because of rounding. 

The estimated five-year total cost of Project Independence per program group 
member was $1,304 and included HRS and non-HRS expenditures. 



ERIC 
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C. Five- Year Net Cost per Program Group Member 



The estimated net cost per program group member for five years is $1,150 (Table 4.6, column 
C). Compared to the two-year net cost of $888 (Table 4.6, column C), the five-year net cost is only 
$262 higher. This small increase in net cost results from the program and control groups' having 
fairly similar patterns of participation in employment-related activities during the projection period, 
with the program group participating in vocational training and post-secondary education at a slightly 
higher rate. Figure 4.4 compares the two-year and five-year costs. The higher five-year total gross 
costs for both the program and control groups are attributable primarily to the cost of additional 
participation in non-Project Independence activities. 

The five-year net cost of $1,150 is one-third of GAIN'S five-year net cost of $3,422. The net 
cost of Project Independence more closely resembles those found in earlier studies of employment 
programs for welfare recipients. 
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^"^For example, in Arkansas WORK, the net cost was Si 18; in Virginia ESP, $430; in San Diego I, $636; 
in San Diego SWIM, $919; and in Baltimore Options, $953 (see Gueron and Pauly, 1991). If the net costs 
are inflation-adjusted to 1993 dollars, both San Diego SWIM and Baltimore Options have a higher net cosi 
than Project Independence. 
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FIGURE 4.4 



ESTIMATED TOTAL GROSS COST PER PROGRAM AND 
CONTROL GROUP MEMBER, AND NET COST PER PROGRAM GROUP MEMBER, 
WITHIN TWO AND FIVE YEARS AFTER RANDOM ASSIGNMENT (IN 1993 DOLLARS) 
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SOURCES: Tables 4.5 and 4.6. 

NOTES: The five-year costs are the sum of the estimated two-year costs plus the projected costs for years 3 to 5. 
Rounding may cause slight discrepencies in calculating sums, averages, and differences. 

(a) As discussed in Chapter 3, contrary to the research design, some control group members received Project Independence 
services. The cost of these services was included in the total gross cost per control group member, 
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CHAPTERS 

THE TWO-YEAR IMPACTS 
OF PROJECT INDEPENDENCE 



This chapter examines whether Project Independence's services and mandates translated into 
changes in the work and welfare behavior of program group members. Their increased participation 
in employment-related activities (relative to the control group) was described in Chapter 3, and die net 
cost of that increased participation was analyzed in Chapter 4. This chapter presents the two-year 
impacts of Project Independence on employment, earnings, AFDC receipt (i.e., the percentage of 
sample members receiving AFDC), AFDC payments (i.e., the dollar amounts). Food Stamp receipt, 
and Food Stamp payments. It also examines patterns of impacts over time; levels of earnings; average 
total income from earnings, AFDC, and Food Stamps; and characteristics of jobs held. The potential 
importance of some control group members having received Project Independence services is also 
analyzed. This chapter focuses on the research sample as a whole, whereas Chapter 6 addresses key 
questions that can be answered only by analyzing the participation and impacts for the early and late 
evaluation cohorts and key subgroups. 

To summarize the major findings of this chapter: Over the two-year follow-up period. Project 
Independence increased employment and earnings and decreased AFDC receipt and payments. The 
program increased (above the control group level) the percentage of program group members ever 
employed during this period by 2.5 percentage points and raised earnings by $227, or 4.1 percent, 
relative to the control group's earnings. Earnings effects were larger in year 1 than year 2 - $146 
versus $80. Over the same period. Project Independence reduced the average length of time on AFDC 
by about three and a half weeks. The program also resulted in a $265 average decrease in AFDC 
payments - a 6.2 percent saving relative to AFDC payments to control group members. The two- 
year reduction in AFDC payments was thus slightly more than the increase in earnings. Two-year 
impacts on Food Stamps were $136 — a reduction of 3.3 percent of the control group average, or 
about half the amount saved in AFDC payments. Program group members experienced a $39 (2.3 
percent) decrease in the sum of their earnings, AFDC payments, and Food Stamps in the last quarter 
of year 2. Responses to the 24-month survey suggest that program group members did not perceive 
any change in the overall quality of their lives as a result of Project Independence. 

Project Independence produced a slight shift in attitudes in the direction of favoring work over 
continued welfare receipt, but the program did not lead to an increase in full-time employment. 
Rather, based on the control group*s employment behavior, it appears to have induced some program 
group members who would not have worked at all in the absence of Project Independence to take part- 
time jobs. These individuals were probably less skilled than other workers: They reported hourly 
wages, weekly hours, and weekly earnings slightly below those of employed control group members. 

This kind of employment did not give program group members a permanent advantage over 
control group members. Employment among the latter increased over time, and by the end of year 
2 had begun to catch up to employment among program group members. As a consequence, the 
earnings difference between the progrrm and control groups declined toward the end of the two-year 
follow-up period. Any future additions to earnings impacts are likely to be smaller than those for the 
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first two years. Impacts on AFDC payments appear to have declined at a slower rate, and future 
AFDC savings are likely. 

The program-control group differences presented in this chapter may underestimate the true 
impact of Project Independence because, contrary to the research design, some control group members 
were made subject to the Project Independence participation mandate and received services through 
the program (as discussed in Chapter 3). This may have reduced the two-year estimates of program 
impacts by as much as 20 percent. Adjusting the impact estimates using this figure, however, would 
not dramatically change the picture of Project Independence's effects for the full evaluation sample. 
Nor would it affect the pattern of declining earnings impacts over time. More important to any 
complete assessment of the program's effectiveness than the issue of Project Independence services to 
controls is the analysis of subgroup impacts undertaken in the next chapter. 

A number of questions remain unanswered in this chapter. Why did earnings impacts decline 
over time? Did the reduction in program dollars and services during the evaluation affect the ability 
of Project Independence to achieve impacts? Were impacts larger during the initial stage of the 
evaluation, when program dollars and staff were larger in proportion to the number of Project 
Independence enrollees? Did the presence of the new mandatory subgroup of AFDC recipients with 
preschool-age children reduce the impacts of Project Independence? Were impacts larger for the 
traditional mandatory subgroup of AFDC recipients who had no preschool-age children? Did Project 
Independence produce larger effects for subgroups the program had targeted for more intensive 
education and training? The answers to these questions have important ramifications for understanding 
the effects of the program, as discussed in the next chapter. 

1. Understanding Impacts 

A. What Do Impact E stimatt^s Mt^an? 

As discussed in Chapter 2, follow-up data for the full research sample are available for each 
of the eight quarters following quarter 1 (the quarter in which each person was randomly assigned to 
the program or control group). Because quarter 1 contains some earnings, AFDC payments, and Food 
Stamp payments from the months and weeks immediately preceding random assignment, it is excluded 
from the summary measures for the first year of follow-up. Hence, the "total first-year" earnings, 
AFDC payments, and Food Stamp payments impacts are defined as the sum of impacts over quarters 
2 through 5; year 2 is defined as quarters 6 through 9. One additional quarter (quarter 10) is available 
for AFDC and Food Stamp data and will be shown graphically where appropriate. 

In analyzing the effects of Project Independence (as well as other welfare-to-work programs), 
a key distinction is between program "outcomes" and "impacts." "Outcomes" refer to the status of 
sample members during one or more quarters in the follow-up period. For this study, four principal 
kinds of outcomes have been measured: employment, earnings, AFDC receipt, and AFDC payments. 
In the following discussion, persons are considered to have been employed in a given quarter if they 
received earnings at any point during that three-month period; similarly, they are classified as having 
received AFDC (or Food Stamps) in a given quarter if they received AFDC (or Food Stamp) payments 
during one or more months within that quarter. 



-87- 



An "impact" is the program's effect on an outcome. The average outcome levels for the 
program group alone provide potentially misleading measures of the impacts of Project Independence. 
Past research has shown that, even in the absence of any intervention, many recipients will go off 
welfare and/or succeed in obtaining a job,^ Thus, even if they had not had access to Project 
Independence, many members of the program group would have reduced their AFDC receipt and/or 
increased their earnings. 

Impacts are estimated by determining the difference between average outcome levels for 
members of the program and control groups. Because individuals were randomly assigned to the 
program and control groups, the two groups should not differ systematically in their background 
characteristics. The only difference between them should be in their access to Project Independence. 
Hence, any differences in outcomes between the two groups can be attributed to Project Independence. 
The differences between program and control group outcomes have been regression-adjusted to control 
for any differences in the two groups' background socioeconomic characteristics remaining after 
randomization, and to increase the precision of the impact estimates.'^ 

B. How Do Impacts Ckcur? How Long Do They Last? 

Welfare-to-work programs can increase employment and earnings in a number of ways. 
Providing job search assistance may help people find job openings or teach them the interview and 
self-presentational skills they need to be hired. The resulting increase in employment clearly increases 
the amount of earnings going to program graduates. Earnings may also be increased by education and 
training, which are intended to build job skills and thereby increase the amount of earnings an 
individual can command. Finally, employment may rise as the result of changes in recipients' attitudes 
or perceptions. For example, exposure to the program may lead to more favorable attitudes toward 
work vis-a-vis continued welfare receipt, thus inducing some people to accept job offers they might 
have rejected previously. Some program enrollees may view the participation requirement as a 
"hassle" and opt for a job to avoid program participation altogether. 

There are three mechanisms through which AFDC receipt and payments could be affected by 
the program. First, by increasing employment rates. Project Independence may reduce AFDC 
payments and rates of AFDC receipt. Since Florida's AFDC grant levels are relatively low, one 



^See Bane and Ellwood, 1983; Ellwood, 1986; and Friedlander, 1988. 

^On a theoretical level, regression adjustment was necessarj' to control for differences in random 
assignment ratios among subgroups and over time — specifically, whether a sample member was randomly 
assigned with a program-control group ratio of 75:25 or 65:35 (the two ratios used in this evaluation). This 
would be standard practice in any experiment having two different program-control group ratios. Beyond that, 
the program and control groups were similar in most background characteristics, but differed in some pre- 
random assignment characteristics that meant that program group members had higher predicted future 
earnings. As a consequence, regression adjustment reduced the estimated program-control group differences 
in outcomes. For earnings, two-year impacts without regression adjustment were $385; controlling for 
random assignment ratio reduced this to $319; controlling for all demographic variables reduced this further 
to the $227 reported in the text. For AFDC payments, two-year savings without regression adjustment were 
$329; controlling for random assignment ratio reduced this to $273; controlling for all demographic variables 
reduced this further to the $265 given in the text. 
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would expect positive impacts on employment to translate quickly into reductions in AFDC receipt.*^ 
Second, regardless of whether Project Independence led to employment gains, the participation 
mandate may have made AFDC unattractive to some members of the program group, leading to 
reductions in their average welfare receipt rate. Third, recipients' AFDC grants are reduced as the 
final step in the sanctioning process for failure to fulfill the Project Independence participation 
requirements. 

Program structure may determine how quickly - or with how much delay - program impacts 
are likely to appear. As discussed in previous chapters. Project Independence was structured to 
increase employment among AFDC recipients as quickly as possible, priiharily through independent 
job search activities. Other studies of programs emphasizing job search assistance have found that 
employment impacts are typically realized within the first year of follow-up - often within the first 
two or three calendar quarters. In those studies, program-control group differences typically continued 
to grow through the end of year 1 and into year 2. Eventually, however, by the third or fourth year 
of follow-up, the employment and AFDC receipt of controls began catching up to that of program 
group members, leading to an eventual narrowing of the differences between the two groups. Thus, 
the results of earlier studies often showed that the annual differences between program and control 
group members grew smaller in succeeding years. Nevertheless, the cumulative impact on earnings 
and AFDC payments kept growing with each additional quarter of follow-up, although at a decreasing 
rate. 

It is often difficult to predict how long program-control group differences in earnings and 
AFDC payments might last, especially for programs that change over time or work with different kinds 
of individuals. In such cases, persistent impacts for one part of the research sample may be obscured 
by weak or declining impacts for another part.^ Analysis of impacts for Project Independence must 
deal with both subgroup differences and programmatic changes over time. These are explored in the 
next chapter. 

C. The Point at Which Random Assi^mnent Takes Place 

In general, impact estimates capture all the effects that have accrued from the time sample 
members were randomly assigned. As discussed in Chapter 1, random assignment in this evaluation 
occurred early in the Project Independence intake process - i.e., at the point of AFDC application 
or eligibility redetermination. Caution is required when comparing the present impacts with those 
found in studies where random assignment was conducted later, during program orienution sessions. 

For two reasons, the earlier point of random assignment may capture impacts that might 
otherwise be missed. First, some persons assigned to the program group may have decided to avoid 
the Project Independence participation mandate by not pursuing their AFDC application, and hence 
would not have received AFDC. Any substantial "deterrence effect"* would have increased overall 
AFDC receipt impacts. Such an effect might also have increased employment and earnings impacts 



^In states with relatively low grant levels, the AFDC eligibility rules make it difficult for persons to 
combine welfare receipt and employment unless their earnings are quite small. 

'^For a case in point, see the comparison of applicant and recipient impacts for the Baltimore Options 
program (Friedlander and Burtless, 1995). 
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if program group members who were deterred from following up on their applications chose instead 
to increase their job search efforts.^ 

Second, the impact estimates include AFDC grant reductions resulting from sanctioning 
program group members for not attending orientation, whereas a research design with a later point of 
random assignment would pick up sanction effects only for orientation attenders who failed to 
participate in post-orientation activities. As discussed in Chapter 3, more than half of those who did 
not attend a Project Independence orientation were referred for a sanction, although it is not clear how 
many of those referrals resulted in an actual grant reduction. 

On the other hand, the Project Independence evaluation design could also make the impact 
estimates appear somewhat smaller than those that might have been found if random assignment had 
been conducted later, at orientation. As noted in Chapter 3, some 23 percent of the program group 
did not attend Project Independence orientation within 12 months after being referred to the program, 
and an additional 34 percent attended orientation but did not participate in post-orientation activities 
(i.e., independent job search, job club, education, or training) during the same follow-up period. If 
participation in these activities, rather than the mandate or sanctioning, is the primary source of the 
program's impacts, then the fact that 57 percent of the program group did not participate may have 
"watered down" the impact estimates. 

D. Inclusion of Nonparticipants. Non-Earners, and Sample Members Off AFDC 

Nevertheless, it is critically important to calculate impact estimates on the basis of all program 
group members, not just those who participated inactivities — especially for mandatory programs such 
as Project Independence, where the very existence of a requirement to participate may produce 
impacts. For example, almost one-quarter of the program group members were referred for a sanction 
at some point during the first year of follow-up because they did not comply with the Project 
Independence mandate. Most of these referrals for a sanction were for people who did not meet the 
very first participation requirement - attending Project Independence orientation - or who attended 
orientation but did not go on to attend an employment-related activity, though required to do so. Such 
effects, which would have been part of the true impact of the program, would not have been captured 
if the impact estimates had been based on orientation attenders or program participants only. Thus, 
for this analysis, program impacts were calculated by including all sample members, nonparticipants 
as well as participants. 

Including all research sample members in the impact calculations means that the estimates of 
average earnings and AFDC payments must be interpreted carefully. For example, estimates of 
average earnings per program group member necessarily will include zero dollar amounts for sample 
members who were not employed during the follow-up period. Similarly, estimates of average AFDC 
payments will include zero dollar amounts for sample members who did not receive AFDC during the 
follow-up period. To the extent that the program converts non-earners to earners, or encourages 



^There is another type of "deterrence" effect that cannot be measured in an evaluation of this kind: 
Knowledge of Project Independence's participation mandate may have discouraged some people from ever 
applying for AFDC or attending a redetermination session. 
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AFDC recipients to leave the rolls, excluding the zero values from the program and control group 
averages would obviously lead to a serious underestimation of program impacts. 

E, Statistical Significance 

Another issue of interpretation concerns the "statistical significance" of impact estimates. 
Statistical significance is a measure of the degree of certainty that some non-zero impact actually 
occurred. If an impact estimate is statistically significant, then one may conclude with some 
confidence that the program really had an effect. If an impact estimate is not statistically significant, 
then the non-zero estimate may be the product of chance. 

Statistical significance does not directly indicate the magnitude or importance of an impact 
estimate, only whether any impact occurred. In an evaluation such as this one, numerically small 
impact estimates are usually not statistically significant. Some numerically large impact estimates may 
not be statistically significant, either, particularly when sample sizes are small. Smaller sample sizes 
yield less reliable impact estimates — estimates in which one can have less confidence — than are 
possible when samples are larger. In this chapter, sample sizes are large. In the next chapter, smaller 
sample sizes are created by breaking up the full sample for subgroup or county-level analyses. 
Therefore, an estimate of a given magnitude that is statistically significant for the full sample will often 
not be statistically significant for a subgroup or a county. 

The next chapter also examines whether impact estimates differed among subgroups and 
counties. In that context, "statistically significant" differences in subgroup or county impact estimates 
give one confidence that the effectiveness of the program was different across subgroups or counties. 
If variation in impacts among subgroups or counties is not statistically significant, then it may be the 
result of chance. In this report, subgroup and county differences in impacts are often not statistically 
significant. 

F, The Effects of Project Independence on Control Group Members 

Part of estimating program impacts for Project Independence is to assess the effect of some 
controls having been exposed to the Project Independence participation mandate and having received 
Project Independence services (an issue discussed in Chapter 3 and Appendix C).^ If a program 
works with some members of the control group, the rate at which those individuals participate in 
employment and training activities is likely to be greater than it otherwise would have been. In 
addition, some of them may be sanctioned for failure to comply with participation requirements. 
Working with some controls may therefore decrease the difference in employment and training activity, 
as well as the difference in sanctioning, between program and control group members. As a 
consequence, the estimated differences in outcomes between the two groups will be less than they 
would have been had no controls been exposed to the program. One might therefore wish to adjust 
the estimated impact upward as a better representation of the true impact of the program, i.e., the 
impact with none of the controls exposed to the program. A correction of the basic impact estimates 
tc take into account participation in Project Independence by some controls is examined in this chapter. 



^Exposure to the program under study by members of the control group is generally referred to among 
researchers as "control crossover." 
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IL The Behavior of the Control Group; The Bench mark fo r Measiiring Impacts 

In a random assignment evaluation, the control group's outcomes ~ such as employment, 
earnings, AFDC receipt, and AFDC payments - are the "benchmarks" used to assess how the 
program affected the program group. A number of studies have shown that the behavior of control 
group members changes over time as a result of normal welfare dynamics. In particular, in a study 
of a large-scale program, control group employment generally increases and AFDC receipt decreases 
substantially following random assignment. These changes occur as some control group members find 
jobs and leave public assistance through employment or other avenues.^ This typical pattern was 
found for the Project Independence control group also. 

The top of Figure 5. 1 displays the control group's average earnings during each of 13 quarters: 
the four quarters prior to random assignment, the quarter of random assignment (quarter 1), and the 
eight quarters of follow-up (quarters 2 through 9). It shows that control group earnings decreased 
dramatically in the quarters leading up to random assignment and then increased gradually over the 
follow-up period. By the last quarter of follow-up (quarter 9), the control group's earnings had risen 
to $849, more than double their earnings of $422 in the quarter of random assignment (quarter 1) and 
about equal to their earnings a year prior to random assignment. In all, control group members earned 
an average of $2,401 during the first year of follow-up (quarters 2 through 5) and $3,138 during the 
second year (quarters 6 through 9). 

The bottom of Figure 5. 1 shows average quarterly AFDC payments for the control group 
(through quarter 10). Average payments were quite low during the four quarters prior to random 
assignment and then increased dramatically in the quarter of random assignment (quarter 1) and the 
first quarter of follow-up (quarter 2). Control group members were receiving an average of $432 in 
AFDC payments by the end of year 2 (quarter 9), 34 percent less than the $654 average for the second 
quarter of follow-up, which was their maximum quarterly payment. In all, control group members 
received an average of $2,348 in AFDC payments during the first year of follow-up (quarters 2 
through 5) and $1,945 during the second year (quarters 6 through 9). 



IIL Two- Year Impacts for the Full Sample 

This section presents Project Independence's two-year impacts on employment, earnings, 
AFDC receipt, AFDC payments. Food Stamp receipt, and Food Stamp payments for the full sample. 
These are presented in Table 5.1 and Figures 5.2 and 5.3. Quarter-by-quarter estimates may be found 
in Appendix Table E.l. 

A, Impacts on Employment and Earnings 

The first section of Table 5.1 shows that, during the follow-up period, 66.3 percent of the 
program group worked, an increase of 2.5 percentage points over the control group rate of 63.8 
percent. The program group also experienced a two-year increase in earnings of $227, a statistically 
significant effect amounting to a 4. 1 percent increase over the control group's earnings of $5,539. A 



^See, e.g.. Bane and Ellwood, 1983; and Friedlander and Burtless. 1995. 
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FIGURE 5.1 



PROJECT INDEPENDENCE: 
QUARTERLY EARNINGS AND AFDC PAYMENTS FOR CONTROL GROUP MEMBERS 
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SOURCES: MDRC calculations from Florida Unemployment Insurance (Ul) eamings records and AFDC records. 
NOTES: The sample used to analyze Project Independence's Impacts was slightly smaller than the full research 
sample. Dollar averages include zero values for sample members who were not employed and for sample 
members who were not receiving welfare. 

Prior quarters are the calendar quarters before the quarter in which random assignment 
occurred (quarter 1). 
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TABLE 5.1 



TWO-YEAR IMPACTS OF PROJECT INDEPENDENCE 
ON EMPLOYMENT. EARNINGS. AFDC RECEIPT. AFDC PAYMENTS. 
FOOD STAMP RECEIPT. AND FOOD STAMP PAYMENTS FOR THE FULL SAMPLE 



Outcome and Foltow-up Period 



Program 
Group 



Control 
Group 



Difference 



Percentage 
Change 



Ever employed (%) 
Years 1 -2 

Year 1 
Year 2 

Last quarter of year 2 

Average total earnings ($) 
Years 1 -2 

Year 1 
Year 2 



66.3 

55.4 
53.3 

38.3 



5766 

2548 
3219 



63.8 

52,6 
51.2 

37.8 



5539 

2401 
3138 



2.5 *** 

2.8 *** 

2 ^ 

0.4 



227 * 

146 
80 



4.0% 

5.4% 
4.1% 

1.1% 



4.1% 

6.1% 
2.6% 



Ever received any AFDC payments (%) 
Years 1 -2 

Last quarter of year 1 
Last quarter of year 2 



88.3 

64.6 
51.2 



89.5 

68.7 
53.6 



-1.1 ** 

—4.2 *** 
—2.4 *** 



-1.3% 

-6.0% 
-4.5% 



Average total AFDC payments received ($) 
Years 1 -2 

Yean 
Year 2 



4028 

2196 
1832 



4293 

2348 
1945 



-265 *** 

-152 *** 
-113 *♦* 



-6.2% 

-6.5% 
-5.8% 



Ever received any Food Stamps (%) 
Years 1 -2 

Last quarter of year 1 
Last quarter of year 2 

Average value of Food Stamps ($) 
Years 1 -2 

Yearl 
Year 2 



90.5 

70.5 
56.8 



4002 

2148 
1855 



91.0 

71.8 
57.4 



4138 

2213 
1925 



-0.5 

-1.3 
-0.6 



-136 

-66 *** 
-70 *** 



-0.6% 

-1.8% 
-1.0% 



-3.3% 

-3.0% 
-3.6% 



Sample size (total = 18.233) 13.509 4.724 

SOURCES: MDRC calculations from Florida Unemployment Insurance (Ul) earnings records, AFDC records, and 
Food Stamp records. 

NOTES: The sample used to analyze Project Independence's impacts was slightly smaller than the full research 
sample. 

Dollar averages include zero values for sample members who were not employed or were not receiving 
welfare. Estimates were regression-adjusted using ordinary least squares, controlling for pre-random assignment 
characteristics of sample members. Rounding may cause slight discrepancies in calculating sums and 
differences. 

For all measures, year 1 refers to follow-up quarters 2-5. and year 2 refers to quarters 6-9. 
Quarter 1 refers toth' luarter in which random assignment occurred. Because quarter 1 may have contained some 
earnings and AFDC payments from the period prior to random assignment, it was excluded from the summary 
measures of follow-up. 

A two-tailed t-test was applied to differences between program and control groups. Statistical 
significance levels are indicated as *** = 1 percent; ** = 5 percent; * = 10 percent. 
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FIGURE 5.2 



PROJECT INDEPENDENCE: 
QUARTERLY EMPLOYMENT AND AFDC RECEIPT RATES FOR THE FULL SAMPLE 
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FIGURE 5.3 



PROJECT INDEPENDENCE: 
QUARTERLY EARNINGS AND AFDC PAYMENTS FOR THE FULL SAMPLE 
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large portion of this gain, $146, was realized during year 1 (see Table 5.1 and Figure 5.3). In year 
2, earnings impacts were down to $80. As shown in Figure 5.2, the control group's employment 
increased over the two years, narrowing the difference from the program group. From a peak in 
quarter 2, quarterly employment impacts declined almost to zero by the final quarter (quarter 9). As 
shown in Figure 5.3, quarterly earnings impacts peaked in quarter 3 and declined thereafter, reaching 
zero in quarter 9. 

Control group members, on average, made $1,950 during each quarter in which they were 
employed (not shown in the table). Program group members made $1,931. The similarity of these 
numbers suggests that the program's effect on earnings is primarily attributable to the increases it 
produced in employment, rather than to its effects on the number of hours worked per quarter or on 
hourly wage rates. This explanation is supported by the findings presented in Table 5.2, which shows 
the distribution of earnings for the program and control groups in year 2. The table shows the 
percentage of program and control group members in six income classes: $0, $l-$999, $l,000-$2,999, 
$3,000-'$6,999, $7,000-$9,999, and $10,000 or more.^ As indicated in the table. Project 
Independence decreased the percentage of program group members with zero earnings, but produced 
little change in the percentage of individuals in each earnings bracket. There was a slight increase in 
the percentage of the sample with the lowest positive earnings ($l-$999 for the year), and a slight 
increase in the second highest bracket ($7,(XX)-$9,999). There was, however, no change in the highest 
earnings bracket. These results support the conclusion that Project Independence increased 
employment bur lid not substantially increase the proportion of workers with higher levels of earnings. 

B. Impacts on AFDC Receipt and AFDC Payments 

Table 5.1 shows that Project Independence reduced the percentage of the program group who 
received any AFDC payments during the follow-up period by 1. 1 percentage points. It also shows that 
Project Independence reduced AFDC payments by an average of $265 over the two-year follow-up 
period, a 6.2 percent decrease from the control group's average of $4,293. AFDC savings were $152 
in year 1 and $1 13 in year 2. The impact in year 2 was large enough to indicate that additional AFDC 
savings will accrue in future years. 

Figure 5.2 and Appendix Table E.l show that quarterly impacts on AFDC receipt peaked in 
quarter 4 at about 5 percentage points, and then declined to about one-third that amount in quarter 10. 
On average, relative to the control group, program group members spent about 0.85 months (about 
three and a half weeks) less on AFDC during the follow-up period. The average AFDC payment per 
month received was $284 for program group members and $285 for control group members. This 
small difference suggests that the decline in the number of months on AFDC, rather than a reduction 
in the average monthly grant for recipients, accounted for most of the reduction in AFDC payments.^ 
As noted above, reductions in receipt were facilitated by Florida^s relatively low benefit levels. 



^The five ranges above zero were selected to correspond approximately to the income distribution of 
employed sample members, about one-fifth of whom had earnings in each of those five ranges (as shown in 
Table 5.2). 

^his inference assumes that the average monthly AFDC grant amount for individuals who left the rolls 
as a result of Project Independence was not very different from that for all sample members who did not leave 
AFDC. 
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TABLE 5.2 

IMPACTS OF PROJECT INDEPENDENCE ON THE DISTRIBUTION OF EARNINGS 
FOR THE FULL SAMPLE IN YEAR 2 



Full Sample : If Ever Employed in Year 2: 

Percentage in Annual Earnings Bracket Percentage in Annual Earnings Bracket (a) 



Year 2 Earnings 
Bracket 


Program 
Group 


Control 
Group 


Difference 


Program 
Group 


Control 
Group 


Difference 


$0 


46.7 


48.8 


-2.1 *** 


n/a 


n/a 


n/a 


$1-$999 


11.4 


10.3 


1.1 ** 


21A 


20,2 


1.2 


$1.000~$2,999 


10.6 


10.4 


0.2 


19.8 


20,2 


-0.4 


$3.000~$6,999 


12.2 


12.4 


-0.2 


22,9 


24.2 


-1.3 


$7,000-$9.999 


7.4 


6.6 


0.8 * 


13,9 ^ 


12.8 


1.1 


$10,000 or more 


11.7 


11.5 


0.2 


21,9 


22.5 


-0.5 



Sample size (total = 18.233) 1 3.509 4.724 

SOURCE: MDRC calculations from Florida Unemployment insurance (UI) earnings records. 

NCTES: See Table 5.1 . 

"N/a" means not applicable. 

(a) Estimates in italics were based only on persons with earnings. Statistical tests were not applied 
to the differences. 
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C. Impacts on Combined Employment and AFDC Receipt 



Table 5.1 shows that the 4.2 percentage point reduction in AFDC receipt in the last quarter 
of year 1 was somewhat larger than the 2.8 percentage point increase in employment for the year. 
Also, the ratio of AFDC impacts to earnings impacts is high in either year, and over the two-year 
follow-up period as a whole. In part, this is because some people left AFDC as a result of Project 
Independence and were not employed. This is illustrated in Table 5.3, which displays the impacts on 
sample members' combined employment and welfare status averaged over year 1 , year 2, and the full 
two-year follow-up period. To create the measures in this table, sample members were classified for 
each quarter into one of four mutually exclusive categories defined by their employment and .\FDC 
receipt status. The percentage of quarters in a year or in the two-year follow-up period spent in each 
status was then calculated for each sample member. For example, someone who spent two quarters 
of the two-year follow-up period in a status would be in the status 2/8 = .25, or 25 percent of the 
time. Impact estimates are the differences between program and control group members. 

The top section of Table. 5.3 shows that Project Independence reduced the percentage of 
program group members who were "not employed and received AFDC" in a quarter by 3.3 percentage 
points, on average, over the two-year follow-up. The effect was 4.0 percentage points in year 1 and 
declined to 2.5 percentage points in year 2. There was very little change in the status "employed and 
received AFDC" in a quarter. Instead, sample members shifted, in approximately equal proportions, 
into the bottom two statuses: There was a 1.8 percentage point increase in "employed and did not 
receive AFDC" and a 1.5 percentage point increase in "not employed and did not receive AFDC." 
Thus, a little less than half the quarters off AFDC produced by Project Independence were quarters 
without employment - representing sample members who were no longer receiving AFDC but did 
not have any earnings to replace those payments. This effect contributed to the high ratio of AFDC 
reductions to earnings gains. 

D. Impacts on Food Stamp Receipt and Food Stamp Payments 

The dollar amount of Food Stamps received by control group members over the two-year 
follow-up period was similar to the dollar amount of AFDC payments: $4,138 for Food Stamps and 
$4,293 for AFDC. The percentage of control group members receiving these two forms of assistance, 
and the amounts received, were also similar in years 1 and 2, with Food Stamp receipt having declined 
at about the same rate as AFDC receipt. 

Project Independence did not affect the number of people who received any Food Stamps at 
all during the two-year follow-up period: 91.0 percent and 90.5 percent of control and program group 
members, respectively. At the end of year 1, there was a 1.3 percentage point decrease in Food 
Stamp receipt, which had diminished by the end of year 2 Project Independence's two-year impacts 
on Food Stamp payments were about half the size of its impacts on AFDC payments, in terms of both 
dollar reductions and the reduction as a percentage of the control group average. As shown in Table 



^Note that sample members who were "not employed and did not receive AFDC" could have received 
support from a variety of sources, including Food Stamps, jobs that were not covered by the Unemployment 
Insurance system (and so were not in the earnings data collected for this report), and contributions from an 
absent co-parent or other family members. 
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TABLE S.3 



TWO-YEAR IMPACTS OF PROJECT INDEPENDENCE ON COMBINED EMPLOYMENT 
AND AFDC RECEIPT STATUS FOR THE FULL SAMPLE 



Outcome and Folbw— up Period 


Program 
Group 


Control 
Group 


Difference 




Percentage of the fo!bw~up period during which 
the sample members were not employed 
and were receiving AFDC (a) 




4^.1 




•kick 


Yean 
Year 2 


48.7 
39.0 


52.8 
41.5 


-4.0 
-2.5 


-k-k-k 
kk-k 


Percentage of the folbw~up period during which 
the sample members were employed 
and were receiving AFDC (a) 
Years 1 ~2 


20.2 


20.2 


-0.0 




Yean 
Year 2 


22.2 
18.2 


21 .8 
18.5 


0.3 
-0.3 




Percentage of the folbw~up period during which 
the sample members were employed 
and were not receiving AFDC (a) 
Years 1 -2 


17.1 


15.3 


1.8 


*** 


Year i 
Year 2 


1 4.6 
19.7 


1 2.7 
18.0 


1 .9 
1.7 


kkk 
kkk 


Percentage of the folbw-up period during which 
the sample members were not employed 
and were not receiving AFDC (a) 
Years 1 -2 


18.8 


17.3 


1.5 


kkk 


Year 1 
Year 2 


14.5 
23.1 


12.7 
22.0 


1.7 
1.2 


*** 


Sample size (total = 18.233) 


13.509 


4.724 







SOURCES: MDRC calculations from Florida Unemployment Insurance (Ul) earnings records and AFDC records. 



NOTES: See Table 5.1. Also, the "differences" are not strictly independent across categories. 

(a) Calculated as the number of quarters in the indicated employment and AFDC receipt status, divided by 
the number of quarters in the follow-up period, and multiplied by 100. 
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5.1, Food Stamp payments were down by $66 per sample member in year 1 and $70 in year 2, for 
a two-year saving of $136, or 3.3 percent of the control group's average payments. These annual 
estimates mask a modest decline in quarter-by-quarter impacts beginning in year 2. The decline was 
not sharp, but the pattern suggests that total Food Stamp impacts will not continue to grow with time. 
Average Food Stamp payments for months in which they were received were $261 and $264 for 
program and control group members, respectively. The difference, which was about LI percent, 
indicates that Food Stamp dollar reductions came mostly from fewer months of receipt and much less 
from reduced monthly payments. 

E. Impacts on Total Income from Earnings^ AFDC. and Food Stamps 

Table 5.4 shows that in the last quarter of follow-up, total earnings, AFDC payments, and 
Food Stamp payments averaged $1,719 per control group member. Program group members averaged 
$39 less, for a decrease of 2.3 percent relative to the control group. About one-fifth of sample 
members in both research groups had no income from earnings, AFDC, or Food Stamps. But they 
may have had income from a spouse, other partner, or family members, or from jobs not covered by 
the Unemployment Insurance system. Project Independence had no impact on the number of people 
who did not have earnings of their own, AFDC, or Food Stamps. 

F. The Effects of Exposure to Project Independence by Some Control Group Members 

Exposure of some control group members to Project Independence may have reduced impact 
estimates by as much as. 20 percent. For example, two-year impacts on earnings may have been as 
high as $283 instead of $227, and two-year AFDC savings may have been as high as $330 instead of 
$265. This section gives the rationale and method for adjusting impact estimates for control group 
exposure to Project Independence. 

In this study, it is estimated (see Chapter 3 and Appendix C) that 19.8 percent of controls were 
exposed to Project Independence services and mandates: An estimated 7.5 percent of controls attended 
Project Independence orientation and participated in Project Independence activities; an estimated 11.1 
percent attended orientation but did not participate in Project Independence employment-related 
activities; and 1.2 percent did not attend an orientation and received a sanction notification. This 19.8 
percent figure is likely an overestimate, and the true crossover rate may well have been lower. 
Exposure of controls to Project Independence services and mandates probably reduced the treatment 
difference between the program and control groups somewhat. If so, then the impact estimates given 
in this study would tend to understate the true impact of the program. In general, when there has 
been control group exposure to the program under study, impact estimates will be affected if two 
conditions hold: (1) Few of those control group members would have obtained similar services 



^^This inference assumes that the average monthly Food Stamp payment for individuals who stopped 
receiving them as a result of Project. Independence was not very different from that for sample members who 
kept on receiving Food Stamps. 

^^Since control group exposure to Project Independence occurred mostly in the first year of follow-up, 
it is unlikely to have contributed to changes in impacts between year 1 and year 2. In addition, exposure rates 
appear to have been similar for control group members in the early and late cohorts, so exposure would not 
have affected differences in impacts for the two cohorts, which are described in Chapter 6. 
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TABLE 5.4 



IMPACTS OF PROJECT INDEPENDENCE ON INCOME FROM EARNINGS. AFDC. AND FOOD STAMPS 
AND ON INCOME SOURCES FOR THE FULL SAMPLE IN THE LAST QUARTER OF YEAR 2 





Program 


Control 




Percentage 


Outcome 


Group 


Group 


Difference 


Change 


Average total value of AFDC, Food Stamps. 










and earnings in quarter 9 ($) 


1680 


1719 


-39 


-2.3% 


Income sources (%) 










Earnings without AFDC 


21.8 


20.2 


1.7 ** 


8.3% 


Earnings and AFDC 


16.4 


17.7 


-1.2 ** 


-7.0% 


AFDC without earnings 


34.8 


36.0 


-1.2 


-3.3% 


No AFDC or earnings 


27.0 


26.2 


0.8 


2.9% 


Food Stamps without AFDC or earnings 


4.1 


3.8 


0.3 


7.2% 


No Food Stamps, AFDC, or earnings 


22.9 


22.4 


0.5 


2.2% 


Sample size (total = 18,233) 


13,509 


4,724 







SOURCES: MDRC calculations from Florida Unemployment Insurance (Ul) earnings records, AFDC records, and 
Food Stamp records. 



NOTES: See Table 5.1 . 
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elsewhere in the community, and (2) the program's services (or mandates) had an impact on the 
control group members who received them.^^ 

Regarding the first condition, it is likely that some of the control group members who were 
exposed to Project Independence would have sought out education and training opportunities on their 
own, as evidenced by the behavior of other controls (as discussed in Chapter 3). Therefore, adjusting 
the impact estimates on the basis of the estimated rate of exposure of controls to Project Independence 
is likely to produce an upper bound of what the correction really should be. In other words, the true 
impact will probably be less than the "corrected" impact. The extent of such overcorrection may be 
substantial.^"^ 

The second condition may or may not hold in this instance. Controls who were exposed to 
Project Independence may well have experienced average impacts of a similar magnitude to the 
average impact for members of the program group. Or their impacts may have been smaller or larger, 
given any differences in characteristics between them and members of the program group. If the 
average impact on a control who was exposed to the program under study is less than the average 



^^It is a common but mistaken belief that exposed controls can simply be dropped from the analysis sample 
without biasing the impact estimates, as long as their observed characteristics are similar, on average, to those 
of the rest of the control group. Differences in wnobservable characteristics, however, can lead to bias if the 
exposed controls are dropped. Also, if exposed controls would have participated in activities on their own 
at rates higher than the average control group participation rate, then dropping them will make the treatment 
difference between program and control groups larger than it really is, thereby biasing the impact estimates 
upward, even if the characteristics of exposed controls are similar to those of other members of the group. 
(Conversely, if controls who are dropped would have participated in activities on their own at a lower rate 
than the average for the control group, the impact estimates could be biased downward.) 

In any event, according to their survey responses, the exposed controls in the Project independence 
research sample differed from other members of the control group only in the extent to which they participated 
in employment and training activities in the period just prior to random assignment. This was determined by 
selecting the conu*ol group members in the 24-month siu^ey subsample and regressing the dummy variable 
"exposed control" on demographic characteristics. The F-value for the regression was statistically significant 
only when a variable was included for "currently enrolled in employment or training activities at the time of 
random assignment." Otherwise, the F-value was not statistically significant, indicating that exposure to 
Project Independence among controls was not strongly related to any of the other characteristics utilized in 
the regression. The same result was found when exposure to Project Independence was defined broadly (as 
having heard any mention that they were required to look for work or to participate in an employment-related 
activity) and more narrowly (as having attended Project Independence orientation, having participated in the 
program, or having been sanctioned for failure to comply with a program requirement). 

^'^Two pieces of evidence suggest that controls exposed to Project Independence would have participated 
in employment- related activities at significant rates on their own, even without the program. First, controls 
who were enrolled in an employment and training activity just prior to random assigtmient had a statistically 
significantly greater probability of subsequently being classified as "exposed to Project Independence. " Those 
individuals might well have continued their activities into the follow-up period even without the program. 
Second, status as an "exposed" control could not be distinguished statistically from status as a "non-exposed" 
control participating in a self-initiated employment and training activity. In other words, in the subsample 
of exposed control group participants and self-initiated, non-exposed control group participants, a regression 
of "exposed to Project Independence" on the list of demographic characteristics produces an F-value that is 
not statistically significant. 
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impact on a program group member, then the correction to the estimated impact will be an 
overcorrection. The opposite will hold if the average impact on a control who was exposed to the 
program is more than the average impact on a program group member. There is, however, no 
evidence to determine which condition pertains. It is therefore reasonable to assume that effects on 
control participants were no greater than effects on participants in the program group. 

Evaluators facing the problem of control group exposure to program services generally adopt 
a standard set of assumptions to obtain corrected impact estinnates. In this instance, those assumptions 
take the following form: that none of the 19.8 percent of controls who were reached by the program 
would have engaged in similar services on their own; that the average impact on them was similar in 
magnitude to the average impact on program group members; and that the program had no impact on 
the other 80.2 percent of controls who were not exposed to the program. Under these assumptions, 
the standard correction to the original impact estimates is made by dividing them by one minus the 
fraction of controls who were exposed to the program. Since these assumptions are not fully 
realized in Project Independence, the resulting estimate will constitute an upper bound on program 
impacts. Using the 19.8 estimate as the basis for this adjustment yields two-year Project Independence 
impacts of $283 (rather than $227) on earnings and $330 (rather than $265) on AFDC savings, and 
the percentage reduction in AFDC payments increases from 6.2 percent to 7.7 percent. Food Stamp 
savings would be $170 (rather than $136) or 4. 1 percent (rather than 3.3 percent) of the control group 
mean. Although the adjustment increases the estimated program impacts, it does not affect the finding 
of a decline in earnings impacts from year 1 to year 2, which, in the absence of actual data beyond 
year 2, is important in projecting what future earnings impacts might be.^^ 



^^The derivation of this adjustment is as follows. Let the true impact per program group member be called 
D. The observed mean outcome for the program group will then be 
Mp = M 4- D, 

where M is the mean outcome in the absence of the program, if no controls aie exposed to the program, then 
the mean outcome for the control group will be 
M, = M. 

If the fraction f of the control group is exposed to the program and obtains the same average impact as 
program group members, then 

M, = M + fxD + (l-f)x0 = M + fxD. 
The impact actually estimated, d, will then be 

d = Mp - M, = (M + D) - (M + fxD) = Dx(I-f), 

yielding 

D = d / (1-f). 

That is, the true impact of the program is equal to the observed impact divided by one minus the crossover 
rate. 

^^^e available data indicate that control group exposure to Project Independence did not increase from 
year 1 to year 2 and therefore did not contribute to the decrease in earnings impacts from year 1 to year 2. 

Another way to assess the importance of program services for control group members is to ask how 
large the impacts of Project Independence on control group members reached by the program would have to 
have been to make a substantial difference in impact estimates. Using the 19.8 percent estimate again, it is 
found that controls reached by Project Independence would require an average impact approximately five times 
the size of the observed program impact in order to make the observed impact half the size of the true 

(continued...) 



-104- 



TV. 



Two- Year Impacts on Job Quality and Qxiality of Life 



The previous sections of this chapter reported that Project Independence increased the 
employment and earnings of the full sample of program group members, but that those increases were 
relatively modest and declined to almost zero by the end of the second year of follow-up. The findings 
also indicate that most of the impact on earnings came from the program group's having had a higher 
rate of employment than the control group, and not higher earnings once employed. The decline in 
earnings impacts over time may have come about because program group members did not obtain 
higher-paying or longer-lasting jobs than control group members. The analysis in the preceding 
section was not able to pursue this hypothesis very far because quarterly Unemployment Insurance (UI) 
earnings data (the data used in that analysis) do not include information about the characteristics of the 
jobs people held. 

Using data for the 1,029 program and control group members who responded to the Project 
Independence 24-month survey, this section examines the characteristics of the jobs held by sample 
members: the hourly or weekly wage rates, number of hours worked per week, and number of weeks 
worked, which combine to make annual earnings. The pattern of impacts on these components of 
earnings provides crucial information about the processes through which Project Independence 
produced the earnings impacts. This section also presents information about selected non-wage 
characteristics of these jobs, such as fringe benefits, job security, and potential for advancement. The 
analysis concludes by examining sample members' perceptions of their quality of life and their attitudes 
toward work and welfare. 

A. Impacts on Job Quality 

The top section of Table 5.5 shows the percentage of program and control group members who 
were ever employed during the two-year survey follow-up period and the percentage who were 
employed at the end of that period. Because survey respondents constituted only part of the total 
impact sample, the estimates for them may differ from those already presented for the full sample. 
Also, survey data about jobs were based on memory rather than records made at the time of 
employment, as is the case with Unemployment Insurance (UI) data.^^ In general, however, results 
from the 24-month survey were quite similar to those presented earlier. The survey data indicate that 
about two-thirds of the program group worked at any time during the follow-up period, and somewhat 
over one-third were employed at the end of that period. Project Independence increased by 4.6 
percentage points the number of sample members who worked at my time during the two years. This 
is a somewhat larger effect than was seen for the full sample earlier. As was the case for the full 



^^(... continued) 

program impact. This figure may be obtained from the equations given in footoote 15 by imposing the 
condition that d = 0.5 x D. The multiple of five is extremely large, suggesting that the observed program 
impacts are well over half the true program impact. 

^^Furthermore, some jobs are not covered by the UI system. For these and other reasons, it is not unusual 
for survey employment and earnings data and UI earnings data to produce different findings on employment 
effects. For a discussion of this issue, see Bloom et al.. 1993. 
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TABLE 5.5 



TWO-YEAR IMPACTS OF PROJECT INDEPENDENCE ON 
SELECTED CHARACTERISTICS OF SAMPLE MEMBERS' MOST RECENT JOB 



Outcome 



Program 
Group (%) 



Control 
Group (%) 



Difference 



Employment 

Ever employed during the 
two year follow-up period 

Employed at the end of the two-year 
follow- up period 



65.6 



38.3 



60.9 



38.9 



4.6 



-0.7 



Weekly hours {a) 

Ever employed during the folbw-up period 
AND most recent job provided: 

Less than 30 hours of work per week 

At least 30 hours of work per week 



21.6 
43.2 



15.7 
44.3 



5.9 ** 

-1.1 



Weekly earnings (b.c) 

Ever employed during the follow-up period 
AND most recent job paid: 

Less than $1 50 per week 

At least $150 per week 



27.1 
37.1 



21.5 
37.3 



5.6 ** 

-0.2 



Health benefits (d) 

Ever employed during the follow-up period 
AND most recent job provided: 

No health benefits 

Health benefits 



44.6 



20.4 



37.4 



23.2 



7.2 



-2.8 



Sample size (total = 1 .029) 



520 



509 



SOURCE: The 24-month survey of a subsample of program and control group members. 

NOTES: Estimates are regression-adjusted using ordinary least squares, controlling for pre-random assignment 
characteristics of sample members. 

Rounding may cause slight discrepancies in calculating sums and differences. 

A two-tailed t-test was applied to the differences between program and control groups. Statistical 
significance levels are indicated as *** = 1 percent;** = 5 percent; * = 10 percent. 

Job characteristics refer to the most recent job as of the time of the 24-month survey interview. Sample 
members not employed during the follow-up are counted as "no's" or zeroes to all outcomes. 

(a) Excludes the less than 3 percent of this sample who did not respond to these survey items. 

(b) Most respondents reported pre-tax earnings. However, a sizable minority (approximately 1 3 percent) 
reported post-tax earnings. No adjustment was made for those reporting post-tax earnings. Therefore, the estimates 
presented in the table slightly underestimate the percentage of respondents with gross weekly earnings in excess of a 
given level. 

(c) Excludes the less than 5 percent of this sample who did not respond to these survey items. 

(d) Excludes the less than 2 percent of this sample who did not respond to these survey items. 
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sample, about equal percentages of program and control group members were still unemployed at the 
end of the follow-up period. 

The remainder of Table 5.5 shows the percentages of program and control group members who 
worked at jobs with certain characteristics associated with job quality. These percentages were based 
on the most recent job for those who held more than one job during the follow-up period. They also 
included zero values for sample members who reported that they did not work at all during the follow- 
up period. By including all program group members and all control group members in the analysis, 
it was possible to determine Project Independence's impact on program group members' likelihood of 
getting a particular type of job, compared to what their chances of obtaining such a job would have 
been in the absence of Project Independence. The analysis focused on three summary characteristics 
of respondents' most recent job: whether the job was part-time (defined as less than 30 hours per 
week) or full-time (defined as at least 30 hours per week); whether the respondent earned less than or 
at least $150 per week; and whether or not the job provided health benefits. The absence of increases 
in full-time employment and weekly earnings corresponds to the lack of impacts on the top bracket of 
annual earnings, the "$10,000 or more" annual earnings category, shown in Table 5.2.^^ 

Table 5.5 shows that Project Independence had virtually no effect on full-time employment, 
but that it did have a statistically significant impact on part-time employment. This means that Project 
Independence had an impact on employment mainly because some program group members who would 
not otherwise have worked took part-time jobs. Those who were working full-time were likely to have 
done so anyway, since they were matched by a similar proportion of control group members who also 
had full-time jobs. In addition. Project Independence did not increase the percentage of program group 
members who worked in jobs that paid at least $150 per week or in jobs that provided health benefits. 
The absence of increases in full-time employment and weekly earnings corresponds to the lack of 
impacts on the top bmcket of annual earnings, the "$10,000 or more" annual earnings category, shown 
in Table 5.2.^^ 

Impacts on Job Characteristics Among Employed Sample Members 

Tables 5.6 and 5.7 use the 24-month survey data to examine characteristics of the most recent 
jobs of program and control group members wfio were employed at some point during the two-year 
follow-up period. Estimates in these tables, unlike those in Table 5.5, do not include program group 
members and control group members who did not work. They therefore do not provide true measures 
of the impact of Project Independence. This is because the personal characteristics of employed 



^^Because the survey subsample was much smaller than the full sample, the "ever employed" impact 
estimates in Table 5.5 were not statistically significant, even though they were larger than the full-sample 
impacts calculated from UI data and presented earlier in the chapter. Smaller samples do not yield the same 
statistical precision as larger samples. This means that impact estimates in smaller samples are more subject 
to chance variation and are less likely to be statistically significant at any given level. 

^^Differences in the number of missing survey responses for the various categories of survey questions 
imply that percentage distributions and other sums will not necessarily precisely match across sections of the 
tables in this portion of the chapter. 

^^mpacts on levels of earnings (shown in Table 5.2) were rerun on the survey sample alone and yielded 
the same pattern as that found for the full impact sample. 
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TABLE 5.6 



WEEKLY HOURS. WEEKLY EARNINGS, AND HOURLY WAGES OF THE MOST RECENT JOB 
AMONG PROJECT INDEPENDENCE SAMPLE MEMBERS WHO WERE EVER EMPLOYED 
WITHIN TWO YEARS AFTER RANDOM ASSIGNMENT 



Outcome 



Employed Program 
Group Members 



Employed Control 
Group Members 



Difference 



Average number of hours 
usually worked per week (a) 

Percentage distribution of number 
of hours worked per week (a) (%) 

Fewer than 10 hours 

10-19 hours 

20-29 hours 

30-39 hours 

40 hours or more 

Average earnings per week (b,c) ("^^ 

Percentage distribution of 
earnings per week (b,c) (%) 

Less than $100 

$100-$149 

$150-$199 

$200 -$299 

$300 or more 

Average earnings per hour (b.d) ($) 

Percentage distribution of 
earnings per hour (b,d) (%) 

$4.25 or less 

$4.26-$5.00 

$5.G1-$6.00 

$6.01 -$7.00 

$7.01-$10.00 

More than $10.00 



32.3 



1.5 
6.6 
23.0 
23.3 
43.3 

176.30 



21.0 
21.0 
21.9 
27.5 
6.6 

5.36 



31.1 
25.2 
19.2 
10.4 
10.4 
3.6 



33.4 



4.0 
6.7 

15.4 
26.5 
47.3 

164.63 



17.8 
16.9 
19.6 
32.7 
11.0 

5.60 



26.7 
24.6 
19.5 
10.1 
11.9 
7.2 



-2.5 (Z) 
2.1 (Z) 
7.6 (z) 
-3 2 (z) 
-4.0 (z) 

-6.33 (z) 



3.2 (Z) 
2.1 (Z) 

2.3 (z) 
-5.3 (z) 
-2.4 (Z) 

-0.42 (z) 



4.4 (z) 
0.6 (Z) 
-0.3 (z) 
0.3 (z) 
'1-5 (z) 
-3.4 (z) 



Sample size (total = 651) 



342 



309 



SOURCE: The 24~month survey of a subsample of program and control group members. 

NOTES: Italics Indicate a nonexperlmental comparison. 
Estimates were not regression -adjusted. 

Rounding may cause slight discrepancies in calculating sums, averages, and differences. 

(a) Excludes the less than 4 percent of this sample who did not respond to these survey items. 

(b) Most respondents reported pre-tax earnings. However, a sizable minority (approximately 13 percent) reported 
post-tax earnings. No adjustment was made for those reporting post -tax earnings. Therefore, the estimates presented 

in the table slightly underestimate the percentage of respondents with gross weekly earnings In excess of a given level. 

(c) Excludes the less than 8 percent of this sample who did not respond to these survey items. 

(d) Excludes the less than 1 0 percent of this sample who did not respond to these survey items, 
(z) Nonexperlmental comparison; no test of statistical significance. 
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TABLE 5.7 



NON-WAGE CHARACTERISTICS OF THE MOST RECENT JOB AMONG 
PROJECT INDEPENDENCE SAMPLE MEMBERS WHO WERE EVER EMPLOYED 
WITHIN TWO YEARS AFTER RANDOM ASSIGNMENT 



Outcome 



Employed Program 
G roup Members (%) 



Employed Control 
Group Members (%) 



Difference 



Number of hours worked 
per week changed "a lot 
or "a fair amount" (a) 

Job provided: 
Pa id s ick days (b) 
Paid vacation days (b) 
Health benefits (a) 
Dental t^nefits (a) 
Tuition assistance or 
paid training classes (a) 

Seasonal job (b) 

On a 0-10 scale, percentage who 
strongly agreed (answered 7- 10) 
with the following statements: 

'The job security was good. " (a) 

"The job had good opportunities 
for promotion or advancement" (a) 

"The people there taught you new 
things that would be valuable for 
doing your job better " (a) 

"The skills you were learning 
would be valuable for getting 
a better job," (b) 

On a 0- 10 scale, percentage who 
reported high job satisfaction 
(answered 7- 10): (a) 



29.9 



30.2 
39.8 
31.6 
19.6 

16.6 
29.6 



50.6 



33.2 



46.2 



45.1 



49.9 



28.8 



27.1 
42.3 
38,0 
27,3 

15.0 

29.4 



51.0 



29.5 



54,0 



52,3 



55.4 



1.1 (2) 

3.0 (2) 
-2.5 (2) 
-6,4 (2) 
-7.7 (2) 

1.6 (2) 
0.2 (2) 



-0.4 (2) 
3.8 (2) 

-5.6 (2) 

-7.2 (2) 

-5.6 (2) 



Sample si2e (total =651) 



342 



309 



SOURCE: The 24-month survey of a subsample of program and control group members. 

NOTES: Italics Indicate a nonexperimental comparison. 
Estimates were not regression-a ijusted. 

Rounding may cause slight discrepancies in calculating sums, averages, and differences. 

(a) Excludes the less than 3 percent of this sample who did not respond to these survey items. 

(b) Excludes the less than 2 percent of this sample who did not respond to these survey items. 
(2) Nonexperimental comparison; no test of statistical significance. 
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program and control group members were not necessarily the same. The similarity of program and 
control groups, assured by the random assignment process that created them, cannot be assured when 
examining just those in each research group who were employed. Tables 5.6 and 5.7 need to be 
interpreted carefully for this reason. The tables indicate whether program group members who worked 
obtained jobs that were better, worse, or comparable to the jobs of control group members who 
worked. But they do not say whether any differences in job characteristics resulted from differences 
in the kinds of people in each research group who worked, or from effects that Project Independence 
might have had on the ability of program group members to find and hold different kinds of jobs. 

The findings presented in Table 5.6 are consistent with those described in the previous section. 
Employed program group members worked an average of 32.3 hours per week at their most recent 
job, while employed control group members worked an average of 33.4 hours. The negative 
difference of 1 . 1 hours represents 3 percent fewer hours of work per week for employed program 
group members. Table 5.6 also shows that the most recent jobs held by employed program group 
members were more likely to have fallen in the low end of the distribution of hours of work per week 
(fewer than 30 hours per week) than the most recent jobs held by employed control group members, 
and were less likely to fall in the high end of the distribution (30 or more hours per week). Weekly 
earnings were 3 percent lower for employed program group members compared to employed control 
group members ($178.30 versus $184.63), and hourly wages were 7 percent lower ($5.38 versus 
$5.80). In the distributions of both weekly and hourly earnings presented in Table 5.6, employed 
program group members were more likely than employed control group members to hold jobs in the 
lower brackets, and less likely to hold jobs in the upper brackets. 

Table 5.7 presents differences between employed program and control group members on five 
types of job benefits and on overall satisfaction with their most recent jobs. It shows that the most 
recent jobs held by employed program group members were less likely to provide health or dental 
benefits than those held by employed control group members. Subjective appraisals of job security 
y/ere similar. Somewhat more employed program group members believed there were promotion 
opportunities, but fewer thought they were acquiring valuable skills on the job. Employed program 
group members were also less likely than employed control group members to report being satisfied 
with tlieir most recent job. 

In summary, the survey findings indicate that Project Independence's strategy of encouraging 
quick entry into the labor market for most of the caseload produced an increase in employment that 
was mainly in part-time, relatively low-paying jobs that did not provide fringe benefits or job 
satisfaction. It appears that the alternative, however, was that the program group members who took 
these jobs would not have worked at all. The lower average job quality among employed program 
group members may have been the result of program group members with marginal job skills having 
been induced to enter the labor market. Poor skills command lower wages. Therefore, the program's 
success in increasing employment for these program group members would have lowered the average 
job quality of those employed without affecting the job quality of persons who would have worked 
even without the program. 

C. Impacts on Quality of Life and Attitudes Toward Work and Welfare 

The analysis of survey data indicated that Project Independence produced little effect on other 
economic and noneconomic aspects of the lives of program group members. On most measures in 
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Table 5.8, program and control group members had quite similar outcomes. This was true, for 
example, for the percentage who were currently married, had recently given birth, were covered by 
Medicaid, were in government-subsidized housing, and were currently satisfied with life as a whole. 
The program group had slightly higher percentages of people who were not covered by health 
insurance or who had a child who was not covered, and a slightly lower percentage of people who 
were currently "satisfied" or "very satisfied" with their overall standard of living. These differences 
were not statistically significant, however. 

Table 5.9 indicates that Project Independence slightly changed program group members' 
attitudes in favor of work over welfare. Fewer program than control group members strongly agreed 
with the statement: "I would only take a full-time job if it paid more than $6 an hour and provided 
medical benefits." Fewer said it was better to stay home and care for a child than to work, if given 
the choice. Both these differences were statistically significant. Smaller and not statistically significant 
differences were found on other survey items. For example, a somewhat larger percentage of program 
than control group members strongly agreed that: "Even a low-paying job is better than being on wel- 
fare. " Moie program group members also strongly agreed that: "It's wrong to stay on AFDC if you 
can get a job, even a job you do not like. " Fewer felt that making mothers on AFDC work was bad 
for their children. These changes in attitudes are consistent with the finding that Project Independence 
increased employment of program group members who would not have worked in the absence of the 
program. It is not clear whether the increase in employment resulted from the change in attitude or 
the attitude change resulted from the experience of working. 

Finally, although there was no difference between program and control groups, it is interesting 
th'it only about one-quarter of each research group reported that, unless a job offered medical benefits 
and paid more money than AFDC, they would rather remain on AFDC. In other words, most 
respondents said it was better to work, even at a low -wage job without full benefits, than to be on 
welfare. 
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TABLE 5.8 

IMPACTS OF PROJECT INDEPENDENCE ON SELECTED NONMONETARY OUTCOMES 



Outcome 



Program 
Group (%) 



Control 
Group (%) 



Difference 



Family status 

Currently married, living with spouse (a) 
Gave birth during the follow -up period (b) 



14.4 
11.5 



14.3 
11.0 



0.1 
0.5 



Medical coverage and 
disability income (a) 

Covered by (or had a spouse or child 
covered by) Medicaid in the month prior 
to the survey interview 

Not personally covered by Medicaid or 
other health insurance in the month prior 
to the survey interview 

Had a child not covered by Medicaid or 
other health insurance in the month 
prior to the survey intemew 

Received Supplemental 
Security Income (SSI) 



72.9 



24.3 



19.1 



3.0 



73.6 



21.4 



17.2 



3.5 



-0.7 



2.9 



1.9 



-0.5 



Government housing 
and energy assistance (a) 

Currently living in public housing or 
receiving a government rent subsidy 
(e.g., "Section 8" housing assistance) 



19.6 



19.5 



0.1 



Perceptions of 
overall quality of life (a ) 

Currently unhappy, sad, or depressed 
'Very often" or 'tairly often" 

Currently "satisfied" or 'Very satisfied" 
with overall standard of living 

Currently "satisfied'' or 'Very 
satisfied" with life as a whole 

Rated life as a whole "a little 
better*' or "much better" than it 
was two years earlier 



42.4 



41.9 



49.4 



61.4 



42.8 



45.7 



50.0 



63.1 



-0.4 



-3,6 



-0.6 



-1.6 



Sample size (total = 1 ,029) 



520 



509 



SOURCE: The 24~month survey of a subsample of program and control group members. 

NOTES: vSee Table 5.5. 

Responses pertain to status or attitude at t^e time of the 24-month survey inten/iew. 

(a) Excludes the less than 1 percent of this sample who did not respond to these survey items. 

(b) Excludes the less than 8 percent of this sample who did not respond to these survey items, 
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TABLE 5.9 

IMPACTS OF PROJECT INDEPENDENCE ON ATTITUDES TOWARD WORK AND WELFARE 



Program Control 
Outcome Group (%) Group (%) Difference 

Strongly agreed that: "Unless a 
job offers me medical benefits 
and more money than AFDC. 

I'd rather be on AFDC." (a) 26.9 25.3 1.5 

Strongly agreed that: "1 would 
only take a full -time job if It 
paid more than $6 an hour and 



provided medical benefits." (b) 


43.1 


48.8 


-5.7 * 


Strongly agreed that: "Even a 
low— paying job is beuer 
than being on welfare." (a) 


59.2 


55.5 


3.6 


otrongly agreed that. MaKing 
mothers receiving AFDC 
work if they don't want to 
is bad for their children." (c) 


18.3 


21 .3 


—3.1 


Strongly agreed that: "If a mother 
nas a choice of working and staying 
home and caring for her children, 
it is better If she stays at home." (d) 


45.1 


51.1 


-6.0 * 


Strongly agreed that: "It's wrong 
to stay on AFDC if you can 
get a job, even a job you 
do not like." (d) 


66.4 


63.9 


2.5 


Percentage of respondents who said they 
would be very likely to take a full -time job if: 








The job paid a little less than AFDC but 
respondent would like the work, (e) 


58.3 


56.9 


1.4 


The job paid a little more than AFDC but 
respondent would not like the work, (a) 


68.1 


67.7 


0.4 


Sample size (total = 1 ,029) 


520 


509 





SOURCE: The 24-month survey of a subsample of program and control group members. 



NOTES: See Table 5.5. 

Responses pertain to status or attitude at the time of the 24-month survey interview. 

(a) Excludes the less than 3 percent of this sample who did not respond to these survey Items. 

(b) Excludes the less than 2 percent of this sample who did not respond to these survey items. 

(c) Excludes the less than 7 percent of this sample who did not respond to these survey items. 

(d) Excludes the less than 5 percent of this sample who did not respond to these survey items. 

(e) Excludes the less than 4 percen* of this sample who did not respond to these survey items. 
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CHAPTER 6 



THE TWO-YEAR PARTICIPATION RATES AND IMPACTS 
OF PROJECT INDEPENDENCE FOR SUBGROUPS 



The preceding chapter left open some central questions about the relative effectiveness of 
Project Independence when operated as intended and after changes were made to deal with rising 
caseloads, and its effectiveness for single parents with preschool-age children — a large group of 
welfare recipients who, under the JOBS legislation, were newly mandatory for such programs. These 
and other questions are investigated in this chapter, which integrates participation and outcome data 
to describe the results of Project Independence for several segments of the evaluation sample. 

To summarize the findings: There is evidence that Project Independence, when operated as 
originally designed, could produce larger earnings impacts than might be concluded from the full- 
sample averages presented in Chapter 5. Specifically, the early cohort ~ who entered the research 
sample when client-to-caseworker ratios were at their intended levels and child care funding was avail- 
able — had earnings impacts larger than those for the late cohort. Also, these impacts did not show 
any clear tendency to decline over time. Impacts on AFDC payments were similar for the early and 
late cohorts, however. 

Two-year earnings impacts were small for the new JOBS-mandatory group (i.e. , single parents 
with preschool-age children), suggesting that they may be more challenging to work with than the 
traditional mandatory group. Again, however, the results for the early cohort were markedly better 
than for the late cohort ~ especially in year 2, when the impacts for the early-cohort sample members 
with preschool-age children equaled those for the early-cohort sample members without preschool-age 
children. For late-cohort sample members with preschool-age children, the earnings impacts were 
actually negative, bringing down the overall program impact on earnings. Even for early-cohort 
sample members with preschool-age children, earnings impacts declined in year 3. There were 
impacts on AFDC payments for subgroups defined by the age of their youngest child, even when 
earnings impacts did not appear. These impacts held up in year 3 for the early cohort without 
preschool-age children, but declined almost to zero for the early cohort with preschool-age children 
- further evidence of the difficulty of achieving lasting results for the new-JOBS mandatory subgroup. 

Earnings impacts emerged, but not until year 2, for the Project Independence "not job-ready" 
subgroup. This is the subgroup to whom Project Independence allocated education and training 
resources, on the premise that their low skills levels might make it difficult for them to find 
employment with job search assistance alone. Their delayed earnings impacts are consistent with the 
widely held view that "human capital investment" (i.e., education and training) increases earning 
power, but that such effects are delayed because of the greater amount of "up-front" time required for 
participation in such activities compared to job search activities. The two-year impacts on AFDC 
payments for the not job-ready subgroup were not particularly large. 

Across subgroups, impacts on earnings and AFDC payments were not closely correlated. 
Earnings impacts of average or above-average magnitude were foimd for some subgroups that had 
relatively small AFDC impacts, and reductions in AFDC payments were found for some subgroups 
that had relatively small earnings impacts. The use of education and training did not appear to have 
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produced larger AFDC payments impacts, although there is some evidence that it may have produced 
larger long-term earnings impacts. 

It should be noted that the evidence on all of these issues is indicative but not conclusive. 
Results from a number of other JOBS evaluations -- results that might buttress the conclusions from 
the Project Independence evaluation - are not yet available. Additional subgroup analyses performed 
for this chapter and an analysis of impacts by county did not yield significant additional information. 

Finally, a comparison was attempted between Project Independence's results and evaluation 
results for pre-JOBS programs of the 1980s and California's JOBS program, the Greater Avenues for 
Independence (GAIN) Program. In order to control for the changed definition of "mandatory" under 
the JOBS program, the comparison focused on the traditional (i.e., pre-JOBS) mandatory group, single 
parents without preschool-age children. For this group. Project Independence was in the middle rank 
of programs in its impacts on earnings and AFDC payments. For the early cohort, which better 
represents Project Independence operating as its planners intended, it appears likely that there will be 
sustained impacts on earnings for the subgroup without preschool-age children. But earnings impacts 
appear less likely to persist for the late-cohort subgroup without preschool-age children, and these 
impacts therefore compare much less favorably with those for otlier programs. AFDC impacts were 
more consistent than earnings impacts across cohorts in the subgroup without preschool-age children. 
The average dollar impact on AFDC payments per sample member was smaller for Project Indepen- 
dence than for some other programs, but the reduction in AFDC as a percentage of payments to the 
control group equaled or exceeded that found for many other programs in large-scale random 
assignment evaluations. 



L Analysis Issues 

There are three statistical issues important to interpreting subgroup impact findings. First, the 
statistical significance of impact estimates is sensitive to sample size. Impact estimates of a given 
magnitude are less likely to be statistically significant when sample sizes are small. For example, a 
$100 impact on earnings that is statistically significant for the full sample may not be statistically 
significant for subgroups. The smaller sample sizes also mean that impact estimates are less precise. 
For a smaller sample, there is a greater chance that the true impact will be somewhat larger or smaller 
than the estimate of that impact. 

Second, some of the demographic data that define the subgroups in this study were unavailable 
for portions of the sample. For example, just under 10 percent of the sample could not be classified 
by age of their youngest child. Sample members who could not be placed in one or another category 
of a subgroup because the relevant data were missing were not included in the analysis of impacts for 
that subgroup. One consequence was to reduce sample sizes somewhat. Another was that the 
weighted average of the impacts for a particular set of subgroups of the full sample was not generally 
equal to the full -sample impacts presented in Chapter 5. 

Third, an examination of subgroup impacts often requires not only an estimate of the impact 
but also an assessment of the magnitude of differences in impacts across subgroups. The dollar 
difference in impacts does not provide all the relevant information, however. In some cases, it is also 
important to consider whether differences in impacts reflect real, underlying differences or could, 
instead, embody a large element of chance. To minimize the element of chance, tests of statistical 
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significance are performed on the differences in two-year impacts across subgroups. Whenever such 
differences are statistically significant, one can have greater confidence that the underlying impacts for 
the subgroups involved are actually different. For example, estimates of $100 and $200 for the 
impacts for two subgroups may represent chance differences in the estimates rather than real 
differences in the underlying impacts unless tlie difference between the $100 and $200 estimates is 
statistically significant. 



IL The Effects of Changes in Project Independence and Growing AFDC Caseloads; A 
Comparison of Impacts for the Early and Late Cohorts 

This section explores the possible effects of changes in Project Independence services and in 
the AFDC environment in which the program operated. As discussed in Chapter 1, Florida's AFDC 
(and Project Independence) caseloads increased markedly between 1990 and 1992 at the same time as 
the state's fiscal circumstances forced HRS to put a freeze on hiring new Project Independence staff. 
Given the relatively fixed number of Project Independence staff, AFDC caseload growth meant an 
increase in the number of program enrollees for each staff member. This increase inhibited staffs 
ability to follow up with many of their clients, which may have reduced the program's ability to 
achieve impacts with them. 

Also during this period, HRS administrators found that they had expended a large share of total 
AFDC child care funds for Project Independence participants. Beginning in January 1991, in order 
to ensure that adequate resources would be available for AFDC recipients entitled to child care,^ 
Project Independence dramatically reduced the availability of child care for those participating in 
program activities.^ The child care restrictions were likely to have had their greatest effect on parents 
with preschool-age children who, by law, could not be required to participate in program activities if 
child care needs presented a barrier.-^ 



^ Those who leave AFDC for employment are entitled to receive subsidized child care for up to 12 months, 
as long as they continue to work. In Florida, ongoing AFDC recipients who are enrolled in Project 
Independence and are employed fewer than 30 hours per week are also entitled to subsidized child care while 
they are working. 

^Restrictions on the availability of subsidized child care for Project Independence participants were 
implemented at different rates and at different levels, depending on the county. However, all counties began 
implementing some cutbacks in January 1991, and the cutbacks expanded thereafter, with subsidized child care 
eventually being virtually eliminated in many counties. Program enrollees who were already participating in 
education or training and leceiving child care at the time these restrictions went into effect were eligible to 
continue receiving child care until the activity ended. Those who subsequently were determined to need 
education or training services could not, in general, be offered subsidized child care services. If they needed 
child care in order to participate in these activities, and could not secure arrangements on their own, they were 
usually excused from participation. Subsidized child care was also drastically reduced for those assigned to 
independent job search and job club activities, altliough there were Project Independence support services 
resources available that staff could use to cover short-term child care needs for some independent job search 
and job club participants. 

^Although individuals were not required to participate if needed child care was not available, case 
managers often encouraged them to make unsubsidizcd informal arrangements with family or friends in order 
to participate in program activities. 
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A, Impacts on Participation 



Table 6.1 presents the participation measures for two random assignment cohorts: sample 
members entering the Project Independence study (i.e., being randomly assigned to the program or 
control group) before January 1991 (the "early cohort") and those entering the study from January 
1991 on (the "late cohort"). These estimates were calculated from the 24-month survey data. Survey 
response rates among sample members sought for interviewing were high - above 80 percent - and 
the total number of respondents was 1,029. The survey data covered both the early and late cohorts 
and the other major subgroups. On the other hand, these data cover only sample members randomly 
assigned in the middle part of the random assignment period, and concentrated in the more densely 
populated areas in each of the nine research counties. The participation estimates calculated from the 
survey data and presented in this chapter are therefore reasonably, but not perfectly, representative of 
the activities of the full impact sample, early and late cohorts, and other major subgroups. See 
Appendix B for a more detailed analysis of the representativeness of the survey. 

Table 6.1 lists the same activities as Table 3.5 and shows total participation in both Project 
Independence activities and activities available from other community sources. Although the sample 
sizes are small, the table shows that overall participation rates for both program and control groups 
were clearly lower in the later cohort than in the earlier cohort. Among those in the early cohort, 70.7 
percent of program group members participated, compared to 58.6 percent of those in the late cohort. 
Participation among control group members also declined, but not as much. The impact on 
participation theicfore declined, from 26.5 percentage points for the early cohort to 21.4 percentage 
points for the late cohort. The impacts on independent job search and job club were not much 
affected, remaining at about 24 percent for both activities for both cohorts. Rather, the overall decline 
was associated with a decline in the impact on education and training participation. In particular, the 
percentage of program group members participating in education and training activities decreased from 
48.0 percent for the early cohort to 36.6 percent for the late cohort, thereby reducing Project 
Independence's impact on participation in these activities by about half (from 15.5 percentage points 
to 7.6 percentage points), mostly because of a decline in impacts on vocational training and post- 
secondary education. This decrease may have resulted, at least in part, from the reduction in child 
care availability for Project Independence participants in education and training activities. Also, as 
caseloads grew and demands on their time increased. Project Independence staff may have referred 
fewer participants to education and training activities, particularly those who had already participated 
in independent job search or job club aciivities. 

B, Impacts on Employment and Earnings 

Follow-up data on earnings and AFDC for the early cohort extend three quarters beyond those 
for the full sample. As shown in Table 6.2, the early cohort's two-year earnings impacts were $439, 
more than four times the $99 earnings impact for the late cohort over the same period. The difference 
between these estimates was not statistically significant, however. Earnings impacts for the early 
cohort did not decline over time. In fact, they remained steady at $216 in year i . $223 in year 2, and 
$277 in year 3 (taken as 4/3 times the sum of earnings in quarters 10 through 1?,). In contrast, the 
late cohort had only a $111 earnings impact for year 1 (not statistically significant), which declined 
to approximately zero in year 2. The difference between the early and late cohorts' two-year earnings 
impacts does not decline when one controls for the fact tliat the cohorts had somewhat different 
demographic characteristics and were distributed differently across counties. 
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TABLE 6.1 



TWO-YEAR IMPACTS OF PROJECT INDEPENDENCE 
ON RATES OF PARTICIPATION IN EMPLOYMENT- RELATED ACTIVITIES, 
BY RANDOM ASSIGNMENT COHORT 



Program Control 
Group Group Percentage 



Subgroup and Participation Measure 


(%) 


(%) 


Difference 


Change 


Early cohort 










Ever participated in any employment- 










related activity (a) 


70.7 


44.2 


26.5 


60.0% 


Ever participated in independent 










job search or job club (a) 


45.8 


21.8 


23.9 


109.6% 


Independent job search 


43.9 


21 .0 




H no CO/ 


Job club 


14.2 


6.8 


7.4 


109.6% 


ever parxiCipaiea in eoucaiion or iraining ^a^ 


Ao.U 




1 O.O 


A7 CO/ 


ABE or GEO 


11.6 


8.3 


3.3 


40.1% 


ESL 


3.6 


4.4 


-0.8 


-18.5% 


Vocational training or post-secondary education 


36.0 


22.8 


13.2 


57.8% 


On-the-job training 


5.3 


4.9 


0.5 


9.9% 


Sample size (total = 431) 


225 


206 






Late cohort 










Ever participated in any employment- 










related activity (a) 


58.6 


37.3 


21.4 


57.3% 


Ever participated in independent 










job search or job club (a) 


40.3 


16.5 


23.8 


144.5% 


Independent job search 


38.7 


15.8 


22.9 


144.5% 


Job club 


12.5 


5.1 


7.4 


144.5% 


Ever participated in education or training (a) 


36.6 


29.0 


7.6 


26.1% 


ABE or GEO 


12.5 


8.3 


4.3 


52.0% 


ESL 


5.4 


2.0 


3.4 


173.7% 


Vocational training or post-secondary education 


24.1 


20.5 


3.6 


17.6% 


On-the— job training 


3.1 


2.0 


1.1 


54.0% 


Sample size {total = 598) 


295 


303 







SOURCE: The 24-month survey of a subsample of program and control group members. 



NOTES: Participation includes Project Independence and non-Project Independence activities. 

Tests of statistical significance between research groups were not performed. 

Rounding may cause slight discrepancies in calculating sums, averages, and differences. 

(a) Individuals could participate in more than one activity during the follow-up period; therefore, the sum of 
percentages in specific activities may exceed the category percentage. 
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TABLE 6.2 



TWO-YEAR IMPACTS OF PROJECT INDEPENDENCE 
ON EMPLOYMENT. EARNINGS, AFDC RECEIPT. AND AFDC PAYMENTS, 
BY RANDOM ASSIGNMENT COHORT 



Subgroup, Outcome, and Follow-up Period 



Program 
Group 



Control 
Group 



Difference 



Percentage 
Change 



Early cohort 

Ever employed (%) 
Years 1 -2 

Last quarter of year 1 
Last quarter of year 2 
Last available quarter of year 3 (a) 

Average total earnings ($) 
Years 1-2 

Yean 
Year 2 
Years (b) 

Ever received any AFDC payments (%) 
Years 1 -2 

Last quarter of year 1 
Last quarter of year 2 
Last quarter of year 3 

Average total AFDC payments received ($) 
Years 1 -2 

Yean 
Year 2 
Year 3 



65.6 

36.0 
36,5 
38.7 



5619 

2499 
3120 
3538 



87,5 

61,3 
55.3 
41.3 



4003 

2115 
1889 
1438 



63.5 

33,7 
36,1 
34,7 



5180 

2283 
2897 
3261 



88,9 

66.3 
58,3 
43.3 



4244 

2240 
2004 
1503 



2.1 

2,3 
0,4 
4,0 



439 

216 
223 
277 



-1,4 

-5.0 
-3.0 
-2.0 



*** 



-241 

-125 
-115 
-65 



*** 



3.3% 

6,8% 
1,1% 
1 1 ,5% 



8,5% 

9.5% 
7,7% 
8.5% 



-1,6% 

-7,5% 
-5,1% 
-4,6% 



-5,7% 

-5.6% 
-5.7% 
-4.3% 



Sample size (total - 7.509) 



5.612 



1,897 



Late cohort 

Ever employed (%) 
Years 1 -2 

Last quarter of year 1 
Last quarter of year 2 
Last available quarter of year 3 

Average total earnings ($) 
Years 1-2 

Yean 
Year 2 
Year 3 

Ever received any AFDC payments (%) 
Years 1-2 

Last quarter of year 1 
Last quarter of year 2 
Last quarter of year 3 

Average total AFDC payments received ($) 
Years 1-2 

Yean 
Year 2 
Years 

Sample size (total = 10.724) 



66,9 

37,3 
39,5 
n/a 



5873 

2584 
3289 
n/a 



89,0 

66,9 
48.3 
n/a 



4043 

2253 
1791 
n/a 



64,0 

35,2 
39.0 
n/a 



5774 

2473 
3301 
n/a 



89,9 

70,6 
50,4 
n/a 



4333 

2426 
1907 
n/a 



2,9 *** 

2.2 ** 

0,5 

n/a 

99 

111 
-12 
n/a 

-1,0 



-3,7 
-2,2 
n/a 



-289 *** 



-173 
-116 
n/a 



4,5% 

6,3% 
1 ,3% 
n/a 



1 ,7% 

4,5% 
-0,4% 
n/a 



-1,1% 

-5.2% 
-4,4% 
n/a 



-6,7% 

-7,1% 
-6,1% 
n/a 



7.897 



2.827 



ERIC 
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(continued) 



TABLE 6.2 (continued) 



SOURCES: MDRC calculations from Florida Unemployment Insurance (Ul) earnings records and AFDC records. 

NOTES: The sample used to analyze Project Independence's impacts was slightly smaller than the full research sample. 

Dollar averages include zero values for sample members who were not employed or were not receiving welfare. 
Estimates were regression-adjusted using ordinary least squares, controlling for pre-random assignment 
characteristics of sample members. Rounding may cause slight discrepancies in calculating sums and 
differences. 

For all measures, year 1 refers to follow-up quarters 2-5; year 2 refers to quarters 6-9; and year 3 refers 
to quarters 10-13. Quarter 1 refers to the quarter In which random assignment occurred. Because quarter 1 may have 
contair^ some earnings and AFDC payments from the period prior to random assignment, it was excluded from the 
summary measures of follow-up. 

A two-tailed t-test was applied to differences between program and control groups. Statistical 
significance levels are indicated as *** = 1 percent; ** = 5 percent; * = 10 percent. 

Differences in two-year impacts on earnings and AFDC payments between cohorts were not statistically 
significant at the 10 percent level. 

"N/a" means not available. 

(a) For the early cohort, AFDC data were available for all of year 3, but earnings data were available only for 
the first three quarters. 

(b) Year 3 earnings equal the sum for quarters 1 0 through 1 2 times 4/3. 
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The quarter-by-quaner impacts for cohorts are shown graphically in Figure 6. 1 . This figure 
shows that early-cohort earnings impacts, after an initial peak in quarter 3, varied around a level well 
above zero, without showing evidence of decline over time. Late-cohort earnings impacts were not 
only smaller to begin with but also declined to about zero by the middle of year 2. 

What explains the differences between cohorts? As shown in Table 6.2, employment impacts 
for the two cohorts were quite similar in years 1 and 2. If anything, initial employment effects were 
slightly larger for the late cohort. But early-cohort program group members increased their 
employment in more stable jobs. For example, by year 2, the percentage of program group members 
who were employed in all four quaiters of the year had increased by a statistically significant 2.1 
percentage points (not shown in the table) for the early cohort, but by only 0.3 percentage poinu (not 
statistically significant) for the late cohort. As a consequence, the impact on total number of quarters 
employed during year 2 was more than twice as large for the early cohort as for the late cohort (0.085 
versus 0.037). 

Thus, the cohort differences in earnings impacts were not the result of differences in rates of 
job-fmding. Rather, the greater impact on employment stability for the early cohort led to a greater 
impact on total time employed, which, in turn, led to a greater impact on total earnings. It may be 
that the greater ability of Project Independence staff to supervise participation in skill-developing 
activities for the early cohort resulted in an increase in their ability to hold stable employment. Such 
a result would be consistent with human capital theory. These results for the late cohort also raise an 
issue regarding the minimum program effort that must be expended to achieve a significant earnings 
impact for a group of program enroUees. It may be that, when program resources are severely 
stretched and spread thin across a large caseload, the amount of attention that can be paid, on average, 
to each enrollee is not sufficient to reach those enrollees who otherwise could be assisted in obtaining 
more stable employment. If resources fall below a certain threshold level, then the "placements" or 
"job entries" recorded by program staff may occur largely in lower-quality jobs."^ It should be noted, 
however, that the pattern of earnings impacts for the late cohort is not t>'pical of the effects of job 
search, even in fairly low-cost programs. In the great majority of previous random assignment studies 
of programs with a major job search component, earnings impacts have grown for two or three follow- 
up years. Also, there are potential explanations of the cohort differences in earnings impacts other 
than changes in Project Independence over time. For example, changing labor market conditions may 
have played a role: Increasing weakness in the labor market may have contributed to the greater 
instability of job-holding for the late cohort. 

C. Impacts on AFDC Payments 

Unlike impacts on earnings, impacts on AFDC payments were quite similar for the two 
cohorts. Two-year AFDC reductions were $241 for the early cohort and $289 for the late cohort, both 
statistically significant. The small cohort difference in two-year AFDC impacts was not statistically 
significant. Year 1 impacts declined somewhat over time for both cohorts, and year 2 savings were 
almost identical: $115 and $1 16, respectively, for the early and late cohorts. Figure 6. 1 shows almost 



"^An example of possibly overstretched resources may be found in the MDRC study of a job search and 
work experience program in Chicago during the 1980s (Friedlander et al,, 1987), 
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FIGURE 6.1 



PROJECT INDEPENDENCE: 
IMPACTS ON QUARTERLY EARNINGS AND AFDC PAYMENTS 
FOR THE FULL SAMPLE AND COHORTS 
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SOURCES AND NOTES: See Table 6.2. 
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overlapping time patterns for AFDC impacts for the two cohorts. Impacts on the percentage receiving 
AFDC, shown in Table 6.2, were also quite similar for the two cohorts.^ 

Since the stretching of Project Independence resources for the late cohort affected the 
availability of child care, it is likely that program impacts for enrollees with preschool-age children 
would have been affected more than those for other enrollees. A full assessment of cohort differences, 
therefore, depends on the analysis of results for subgroups defined by the age of the youngest child, 
which are reported in the next section. 



III. Impacts for Single Parents Who Had Preschool-Age Childbren 

As previously discussed, the impact results by the age of the youngest child are particularly 
important, given the fact that the Family Support Act of 1988 mandated for the first time at the federal 
level that single parents with children between the ages of three and five (and as young as age 1 , at 
state option) must participate in welfare -to-work programs. A key policy question is whether JOBS 
programs can generate significant earnings or AFDC impacts for single parents with children under 
the age of six, given the special child care needs of preschool-age children.^ Results for the subgroup 
with preschool-age children are also important in light of the change in availability of child care over 
time: Those results may help explain the differences in impacts observed for the early and late cohorts. 

A. Impacts on Participation 

The top section of Table 6.3 displays estimates of participation (combining Project Indepen- 
dence and non-FYoject Independence activities) for sample members whose youngest child was six or 



^Why should AFDC impacts be so similar when earnings impacts were so dissimilar? The answer may 
lie, again, in the kinds of employment impacts achieved for the two cohorts. Although not shown in Table 
6.2, the impacts on ever employed in year 1 were similar across cohorts, and the differences emerged only 
in year 2, from the greater increase in stability of employment for the early cohort. AFDC reductions, 
however, may stem more from the initial push into employment, which is sufficient to speed up AFDC case 
closure, than from greater employment stability over the longer term. Once off AFDC, many former 
recipients do not return quickly to public assistance, even if they lose a job. This leads, as reported in the 
previous chapter, to some increase in time spent off AFDC^ without employment. Such behavior would 
explain the patterns of earnings gains and AFDC reductions for the two cohorts. It also suggests a possible 
trade-off of savings for government budgets against income gains for program enrollees, a trade-off that has 
been noted in other studies. (See especially Friedlander and Burtless, 1995; and Friedlander and Gueron, 
1992.) In particular, it may be possible to achieve some budgetary savings with relatively inexpensive 
services that speed up job- finding and produce quick case closures. Income gains for program enrollees, on 
the other hand, may come more from long-term increases in employment stability and earnings on the job. 

^Subsequent to the launching of the Project Independence evaluation, some impact estimates became avail- 
able for AFDC recipients with children between tlie ages of three and five in three counties that were part of 
the random assignment evaluation of California's GAIN program. In general, the impacts for that subgroup 
in each county paralleled the impacts obtained in the same county for the subgroup with older children. Over 
a three-year follow-up period, one county obtained large impacts on earnings and AFDC payments, another 
produced large (but not statistically significant) impacts on earnings, and the third produced no impacts on 
either outcome. For details, see Riccio, Friedlander, and Freedman, 1994. 
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TABLE 6.3 



TWO-YEAR IMPACTS OF PROJECT INDEPENDENCE 
ON RATES OF PARTICIPATION IN EMPLOYMENT- RELATED ACTIVITIES. 
BY AGE OF YOUNGEST CHILD 





Program 


Control 








Group 


Group 




Percentage 


Subgroup and Participation Measure 


(%) 


(%) 


Difference 


Change 


Tounv^csi cniiu» <ic]6 o or oicisr 










Ever part ici pelted in any employment— 










related activity (a) 


62,4 


41,8 


20.6 


49.3% 


Ever participated in independent 










job search or job club (a) 


43.4 


22.1 


21.3 


96.3% 


Independent job search 


41.6 


21.2 


20.4 


96,3% 


Job club 


13.4 


6.8 


6.6 


96.3% 


Ever participated in education or training (a) 


42.3 


30.9 


11.4 


36.8% 


ABE or GED 


12.9 


8.0 


4.9 


60.6% 




«^ n 

\^ .\j 


o.o 


1 A 


oy. 1 /o 


Vocational training or post— secondary education 


28.7 


23.3 


5.4 


23.1% 


On-the-job training 


47 


2.0 


2.7 


131.8% 


Sample size (total = 528) 


279 


249 






Younqest child, age 3-5 










Ever participated in any employment- 










related activity (a) 


68.2 


38.5 


29.7 


77.1% 


Ever participated in independent 










job search or job club (a) 


45.5 


15.0 


30.4 


202.2% 


Independent job search 


43.6 


14.4 


29.2 


202.2% 


Job club 


14.1 


4.7 


9.4 


202.2% 


Ever participated in education or training (a) 


40.9 


31.0 


9.9 


32.1% 


ABE or GED 


11.6 


9.3 


2.3 


25.1% 


ESL 


5.1 


1.8 


3.3 


185.3% 


Vocational training or post -secondary education 


29.3 


19.5 


98 


50.4% 


On-the-job training 


3.5 


4.4 


-0.9 


-19.9% 


Sample size (total = 424) 


198 


226 







SOURCE: The 24-month survey of a subsample of program and control group members. 



NOTES: Participation includes Project Independence and non-Project Independence activities. 

Tests of statistical significance between research groups were not performed. 

Rounding may cause slight discrepancies in calculating sums, averages, and differences. 

(a) Individuals could participate in more than one activity during the follow-up period; therefore, the sum of 
percentages in specific activities may exceed the category percentage. 
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older, and the bottom section presents the corresponding estimates for those with preschool-age 
children. Project Independence's impact on overall participation rates was somewhat larger for sample 
members with preschool-age children than it was for those with older children. The largest difference 
occurred for independent job search and job club activities, where the impact for those with 
preschool-age children was 30.4 percentage points, compared to 21.3 percentage points for those 
whose youngest child was six or older. This difference was due primarily to the difference in control 
group participation rates in these activities. While 22.1 percent of the control group members whose 
youngest child was six or older participated in independent job search or job club activities, only 15.0 
percent of control group members with preschool-age children did so. The lower participation rates 
for control group members with preschool-age children (and thus the larger impact on participation 
rates) may have resulted from difficulties they had in meeting their child care needs on a short-term 
basis without the help of Project Independence. In fact. Project Independence was able to provide 
short-term child care tor some individuals enrolled in independent job search or job club activities even 
when child care resources for those in education and training activities were limited or eliminated. 

Table 6.3 also shows that Project Independence's impacts on participation in education and 
training activities were somewhat larger for sample members whose youngest child was six or older 
(11.4 percentage points), compared to those with preschool-age children (9.9 percentage points). The 
exception to this is the larger impact on participation in vocational training and post-secondary 
education for those with preschool-age children (9.8 percentage points versus 5.4 percentage points 
for those whose youngest child was six or older). Again, this resulted primarily from the lower 
participation rate among control group members with preschool-age children compared to control group 
members whose youngest child was six or older. 

B. TiTtparts on Employment and Earnings 

Table 6.4 presents the two-year impacts on employment, earnings, AFDC receipt, and AFDC 
payments for the two subgroups defined by the age of the youngest child. ^ For the subgroup without 
preschool-age children, the impact on employment over the two-year follow-up was 4.2 percentage 
points. Employment impacts were found in years 1 and 2, but employment among program and 
control group members had converged by the last quarter of year 2, leaving only a small difference. 
Twoyear earnings impacts totaled $473 per sample member, an increase of 8.2 percent over the 
average for the control group. Earnings impacts were larger in year 1 than in year 2, but year 2 
impacts accounted for nearly 40 percent of the two-year total. It may be expected that total earrings 
gains will continue to accumulate beyond year 2. 

The picture was quite different for sample members with a child three to five years old. There 
was a 3.1 percentage point employment impact in year 1, not much lower than that for tlie subgroup 
without young children. But this effect had largely dissipated by year 2. The measure ''ever employed 
over the full two-year follow-up period" increased only slightly, indicating that the employment 
increase in year 1 represented faster job-finding among sample members who would eventually have 
found work even without Project Independence. Earnings impacts were quite small and not statistically 
significanr in year 1 and they declined in year 2, producing a two-year total earnings increase of only 



It should be noted that there were subst;^ntially smaller earnings impacts for the group of sample members 
who could not be classified according to the age of their youngest child. This implies that the impacts for one 
or both subgroups defined by the age of the youngest child would have been somewhat smaller if the full 
sample had had no missing data on the age of the youngest child. 
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TABLE 6,4 



TWO-YEAR IMPACTS OF PROJECT INDEPENDENCE 
ON EMPLOYMENT. EARNINGS. AFDC RECEIPT. AND AFDC PAYMENTS, 
BY AGE OF YOUNGEST CHILD 



Subgroup. Outcome, and Follow-up Period 



Program 
Group 



Control 
Group 



Difference 



Percentage 
Change 



Youngest child, age 6 or older 

Ever employed (%) 
Years 1 -2 

Year 1 
Year 2 

Last quarter of year 2 

Average total earnings ($) 
Years 1 ~2 

Yean 
Year 2 

Ever received any AFDC payments (%) 
Years 1 -2 

Last quarter of year 1 
Last quarter of year 2 

Average total AFDC payments received {$) 
Years 1 -2 

Year1 
Year 2 



67.1 

56.7 
54,0 

39.1 



6228 

2790 
3438 



87,5 

61,4 
48.3 



3720 

2053 
1667 



62,9 

52,6 
50,0 

37.4 



5755 

2506 
3249 



89,1 

66,8 
51.3 



4024 

2224 
1800 



4.2 *** 

4,1 *** 
4.0 *** 

1.7 



473 ** 

284 *** 

190 



-1.6 ** 

-5.4 *** 
-3.0 *** 



-304 *** 

-171 *** 
-133 *** 



6.7% 

7.8% 
8.0% 

4.6% 



8.2% 

1 1 .3% 
5.8% 



-1.8% 

-8.1% 
-5.9% 



-7.5% 

-7.7% 
-7.4% 



Sample size (total = 9.31 2) 



6.926 



2.386 



Youngest child, age 3—5 

Ever employed (%) 
Years 1 -2 

Year 1 • 
Year 2 

Last quarter of year 2 

Average total earnings ($) 
Years 1 -2 

Yean 
Year 2 

Ever received any AFDC payments (%) 
Years 1 -2 

Last quarter of year 1 
Last quarter of year 2 

Average total AFDC payments received ($) 
Years 1 -2 

Yearl 
Year 2 



65.9 

54.8 
52.3 

37.0 



5202 

2268 
2935 



90.5 

68.6 
55.6 



4467 

2405 
2062 



64.1 

51.7 
51.8 

38.4 



5183 

2214 
2969 



90.6 

71.3 
57.2 



4677 

2538 
2139 



1.8 

3.1 ** 
0.5 

-1.4 ■ 



20 

54 
-34 



-0.4 

-2.7 ** 
-1.6 



-210 *** 

-133 *** 
-77 * 



2.8% 

6.1% 
0.9% 

-3.6% 



0.4% 

2.4% 
-1.1% 



-0.4% 

-3.7% 
-2.9% 



-4.5% 

-5.2% 
-3.6% 



Sample size (total = 7.21 1) 



5.321 



1.890 



SOURCES: MDRC calculations from Florida Unemployment Insurance (Ul) earnings records and AFDC records. 



NOTES: See Table 6.2. 

Differences in two-year impacts on earnings and AFDC payments betv\/een child -age subgroups were 
statistically significant at the 10 percent level. 
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$20, which was not statistically significant and was only 0.4 percent of average earnings for control 
group members in this subgroup. It should be added, however, that the difference in total earnings 
impacts for the two subgroups was not statistically significant, indicating that there is some uncertainty 
about whether real differences betu'een the subgroups do exist. 

As suggested above, one potential explanation for the small earnings impacts for those with 
younger children is that this subgroup encountered more formidable labor market barriers as a result 
of their need for child care. If child care barriers were pervasive throughout this subgroup, however, 
one would expect the average earnings for control group members with children three to five years 
old to have been considerably smaller than those of their counterparts with older children. Although 
there was a difference between these groups' average earnings, it was not very large: Control group 
members whose youngest child was six years old or older had average earnings of $5,755 during the 
two-year follow-up period compared to $5,183 for the group with preschool-age children (see Table 
6.4). This suggests that child care was not a severe barrier to employment for at least some of the 
single parents with preschool-age children. Child care may nevertheless have been a severe barrier 
for many others in tihiis subgroup. As discussed below, this barrier may have been felt most strongly 
by those in the late cohort, when child care availability was constrained. 

C. Impacts on AFDC Receipt and Payments 

Child care barriers could also lead to longer periods of welfare receipt among single parents 
with preschool-age children. There were moderate differences in the levels of AFDC payments for 
the two control groups defined by the age of the youngest child. Whereas control group members with 
no preschool-age children received $4,024 during the two years of follow-up, the comparable figure 
for those with preschool-age children was $4,677. The latter subgroup was more likely to remain on 
AFDC longer, but the differences were moderate rather than large: 57.2 percent were still on aid in 
the last quarter of year 2, compared to 5L3 percent for the subgroup with no young children. 

The two-year AFDC savings for those with preschool-age children were somewhat smaller than 
those for program group members with no preschool-age children: $210 and $304, respectively. Both 
of these impacts were statistically significant (see Table 6.4), and the difference between them was not 
statistically significant. For the subgroup without preschool-age children, there was a 1.6 percentage 
point decrease (statistically significant) in the "ever received AFDC" rate for the two years, indicating 
a slight reduction in the completion of AFDC applications, but this effect was not found for the 
subgroup with pre:School-age children. In addition, impacts on AFDC payments declined from year 
1 to year 2 at a fas.ter rate for the subgroup with preschool-age children. Although cumulative AFDC 
savings appear likely to continue to increase beyond year 2, the increases will probably be larger for 
the subgroup without preschool-age children. 

D. Parents of Prescbool-Age Children in the Early and Late Cohorts 

Table 6.5 presents impacts on participation in employment-related activities for lixe two age-of- 
youngest-child subgroups within the early and late cohorts. It shows that the impact on participation 
for those whose youngest child was age six or older remained stable across cohorts at about 20.5 
percent. For those with preschool-age children, however, the program's impact on participation in 
any employment-related activity decl- led from 35.4 percentage points for the early cohort to 24.4 
percentage points for the late cohor.. For this subgroup, impacts on participation in independent job 
search or job club activities v/er^ ^ne same in both cohorts, but the impact on participation in education 
and training activities declined from 18.5 percentage points for the early cohort to 2.7 percentage 
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TABLE 6.5 



TWO-YEAR IMPACTS OF PROJECT INDEPENDENCE 
ON RATES OF PARTICIPATION IN EMPLOYMENT- RELATED ACTIVITIES. 
BY AGE OF YOUNGEST CHILD AND RANDOM ASSIGNMENT COHORT 



Subgroup and Participation Measure 



Program 
Group 
(%) 



Youngest child, age 6 or older: 

Early cohort Sample size (total = 219) 

Ever participated in any ennployment- 
related activity (a) 

Ever participated in independent 
job search or job club (a) 

Independent job search 

Job club 

Ever participated In education or training (a) 
ABE or GED 
ESL 

Vocational training or post-secondary education 
On-the-job training 

Late cohort Sample size (total = 309) 

Ever participated in any employment- 
related activity (a) 

Ever participated in independent 
job search or job club (a) 

Independent job search 

Job club 

Ever participated in education or training (a) 
ABE or GED 
ESL 

Vocational training or post -secondary education 
On-the-job training 



Control 
Group 



Difference 



Percentage 
Change 



65.6 


45.0 


20.6 


45.8% 


45.4 


23.0 


22.4 


97.3% 


43.6 


22.1 


21.5 


97.3% 




7 1 


A Q 




44.5 


33.0 


11.5 


35.0% 


12.6 


7.0 


5.6 


80.1% 


1.7 


6.0 


-4.3 


-72.0% 


33.6 


25.0 


6.6 


34.4% 


6.7 


3.0 


3.7 


124.0% 


60.0 


• 

39.6 


20.4 


51.5% 


41.9 


21.5 


20.4 


95.0% 


40.2 


20.6 


19.6 


95.0% 


13.0 


6.7 


6.3 


95.0% 


40.6 


29.5 


11.1 


37.6% 


13.1 


8.7 


4.4 


50.6% 


7.5 


2.0 


5.5 


273.1% 


25.0 


22.2 


2.9 


12.9% 


3.1 


1.3 


1.8 


133.6% 



Youngest chiid. age 3-5: 

Eartv cohort Sample size (total = 182) 

Ever participated in any employment- 
related activity (a) 

Ever participated in independent 
job search or job club (a) 

Independent job search 

Job club 

Ever participated in education or training (a) 
ABE or GED 
ESL 

Vocational training or post -secondary education 
On-the-job training 

Late cohort Sample size (total = 242) 

Ever participated in any employment- 
related activity (a) 

Ever participated in independent 
job search or job club (a) 

Independent job search 

Job club 

Ever participated in education or training (a) 
ABE or GED 
ESL 

Vocational training or post -secondary education 
On-the-job training 



77.8 


42.4 


35.4 


83.5% 


50.0 


20.7 


29.4 


142.1% 


48.0 


19.8 


28.2 


142.1% 


15.5 


6.4 


9.1 


142.1% 


50.0 


31.5 


18.5 


58.6% 


10.0 


9.8 


0.2 


2.2% 


6.7 


1.1 


5.6 


51 1 .9% 


36,7 


19.6 


17.1 


87.4% 


3.3 


6.5 


-3.2 


-48.9% 


60.2 


35.8 


24.4 


68.0% 


41.7 


11.2 


30.5 


272.4% 


40.0 


10.7 


29.3 


272.4% 


12.9 


3.5 


9.4 


272.4% 


33.3 


30.6 


2.7 


8.9% 


13.0 


9.0 


4.0 


44.6% 


3.7 


2.2 


1.5 


65.2% 


23.2 


19.4 


3.8 


19.3% 


3.7 


3.0 


0.7 


23.7% 
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(continued) 



TABLE 6.5 (continued) 



SOURCE: The 24-month survey of a subsample of program and control group members. 

NOTES: Participation includes Project Independence and non-Project Independence activities. 

Tests of statistical significance between research groups were not performed. 

Rounding may cause slight discrepancies in calculating sums, averages, and differences. 

(a) Individuals could participate in more than one activity during the follow-up period; therefore, the sum 
of the percentages in specific activities may exceed the category percentage. 
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points for the late cohort. At the same time, the impact on participation in education and training 
activities for those whose youngest child was age six or older remained at just over 1 1 percentage 
points in both cohorts. 

These estimates suggest that Project Independence child care enabled the new JOBS- 
mandatories to achieve a large impact on participation in the longer program activities, namely, 
education and training. Reduced availability' of Project Independence child care for the late cohort 
inhibited their ability to participate in those activities. But were these dramatic differences in 
participation associated with differences in impacts and earnings and AFDC payments? Table 6.6 
displays impacts for the age-of-youngest-child subgroups by cohort. 

The table indicates that the two-year earnings impact for those with preschool-age children was 
larger in the early cohort than in the late cohort. The inter-cohort differences were most apparent 
during year 2, when the subgroup with preschool-age children in the early cohort had an earnings 
impact of $216; the comparable figure for the same subgroup in the late cohort was -$189. The early 
cohort experienced an increase in earnings impacts from year 1 to year 2 (although neither impact was 
statistically significant). By year 3, however, earnings impacts for the early cohort of the subgroup 
with preschool-age children showed a marked decline. The earnings impact in year 3 was less than 
half what it had been in year 2. T?iis decline appears to have been at least partly associated with 
control group "catch-up," i.e., the gradual increase in earnings for the control group over time. For 
AFDC payments, impacts had declined almost to zero by year 3. This pattern of growth and then 
decline of impacts is evidence of the challenge of achieving lasting impacts with the new JOBS- 
mandatory subgroup. 

For the subgroup with no preschool-age children, none of the differences in early-cohort versus 
late-cohort earnings impacts was very large, indicating that the change in availability of child care did 
not affect this subgroup. Moreover, third-year impact estimates for the early cohort of this subgroup 
suggest that program effects may be more persistent than is indicated by the two-year data alone. 
Earnings impacts for the early cohort in year 3 were $424 (4/3 times earnings for quarters 10 through 
12), a statistically significant increase and more than double the earnings impacts of year 2. Why 
earnings impacts should have shown a slight decline from year 1 to year 2 is unclear, but the quarter- 
by-quarter pattern of earnings results (not shown) indicates that earnings impacts are likely to continue 
for this subgroup in the early cohort beyond year 3. The apparent persistence of earnings impacts for 
the early-cohort subgroup with no preschool-age children is typical of the pattern of impacts found in 
prior studies of programs featuring job search and working with this same subgroup. The reductions 
in AFDC payments in year 2 continued in year 3. They held at $101 (statistically significant), almost 
the same as in year 2, although the quarter-by-quarter results (not shown) indicate that these impacts 
are Hkely to be lower in year 4 and beyond. 



rV. Impacts for Subgroups Defined by Job-Readiness Status and Welfare History 
A. Impacts on Subgroups Defined bv Job-Readiness Status 

Project Independence classified new enrollees as either "job-ready" or not. The small subgroup 
classified as not job-ready was not required to participate in independent job search and job club as 
their first activity in order to give them an opportunity to participate in education or training activities. 
The effects of this treatment difference, in combination with differences in socioeconomic 
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TABLE 6.6 



TWO-YEAR IMPACTS OF PROJECT INDEPENDENCE ON EARNINGS AND AFDC PAYMENTS, 
BY AGE OF YOUNGEST CHILD AND RANDOM ASSIGNMENT COHORT 



Average Total Earnings 



Subgroup 

and Fo i(ow-up Period 

You ngest child , age 6 or older: 
Early cohort 



Percentage of 
Sample__ 



23.1 



Program Control 

Group Group Difference Percentage 
($) 1$) ($) Change 



Average Total AFDC Paynr^ents 



Program 
Group 
($1. 



Control 

Group 
($L 



Difference 
(IL 



Percentage 
Change 



Years 1 -2 




6010 


5521 


489 


8.9% 


3641 


3885 


-244 ** 


-6.3% 


Yean 
Year 2 
Year 3 (a) 




2714 
3296 
3668 


2436 
3085 
3244 


278 * 

211 

424 ** 


1 1 .4% 
6.8% 
13.1% 


1943 
1698 
1254 


2082 
1803 
1355 


-139 
-105 * 
-101 * 


-6.7% 
-5.8% 
-7.5% 


Last available quarter of year 3 (b) 




949 


825 


124 ** 


15.0% 


281 


303 


-22 


-7.1% 


Late cohort 


33.3 


















Years 1 -2 




6386 


5899 


487 * 


8.3% 


3775 


4122 


-347 *** 


-8.4% 


Year 1 
Year 2 




2848 
3538 


2541 
3358 


307 ** 
181 


12.1% 
5.4% 


2130 
1645 


2321 
1800 


-192 *** 
-155 


-8.3% 
-8.6% 


Last quarter of year 2 


- ■ 


917 




913 




4 


0.5% 


347 


375 


-28 ** 


-7.6% 


YpurigesjLchild, age 3-5: 




















Early cohort 


19.0 


















Years 1 -2 




5197 


4839 


358 


7.4% 


4495 


4698 


-203 * 


-4.3% 


Yean 
Year 2 
Year 3 (a) 




2271 
2927 
3355 


2128 
2711 
3257 


142 
216 
97 


6.7% 
8.0% 
3.0% 


2364 
2131 
1670 


2450 
2248 
1673 


-86 
"-117 * 
-2 


-3.5% 
-5.2% 
-0.1% 


Last available quarter of year 3 (b) 




872 


859 


13 


1 .5% 


384 


385 


-0 


-0.1% 


Late cohort 


24.6 


















Years 1 -2 




5220 


5395 


-175 


-3.2% 


4439 


4679 


—241 *** 


-5.1% 


Year 1 
Year 2 




2272 
2948 


2258 
3137 


14 
-189 


0.6% 
-6.0% 


2433 
2006 


2614 
2065 


-182 
-59 


-7.0% 
-2.8% 


Last quarter of year 2 




775 


844 


-69 


-8.2% 


431 


445 


-14 


-3.1% 



SOUF^CES: MDRC calculations from Florida Unemployment Insurance (Ul) earnings records and AFDC records. 
NOTES: See Table 6.2. 

Variation in tv^o-year impacts on earnings and AFDC payments across the four cohort/subgroup categories was not statistically significant at 
the 10 percent level. 

(a) Year 3 earnliigs equal the sum for quarters 10 through 12 times 4/3. 
Q (b) For the early cohort. AFDC data were available for all of year 3, but earnings data were available only for the first three quarters. 
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characteristics (see Chapter 2, Section IIC), may have led to different activity rates and different 
impacts for the job-ready and not job-ready subgroups. This section analyzes such differences. 

During the period of random assignment, a program enroUee was judged job-ready if she had 
attained the tenth grade in school or had worked in at least 12 of the previous 36 months. As 
discussed in Chapter 1, however. Project Independence later changed its job-readiness definition to let 
more enrollees participate in education and training activities.^ An important rationale for this 
modification was that many of the job-ready participants (particularly those with no high school 
diploma or GED) may not have been served effectively by the program's up-front emphasis on 
independent job search. 

Table 6.7 presents the findings on Project Independence's impact on participation in 
employment-related activities for three subgroups defined by combinations of the original and modified 
job-readiness criteria.^ The top panel includes those who would be defined as job- ready using either 
set of criteria. This group is referred to in this analysis as the "job-ready subgroup." The middle 
panel includes those who would be defined as job-ready using title original criteria, but would be 
defined as not job-ready using tihe new criteria because they had no high school diploma or GED. 
This group is referred to in this analysis as the "modified job-ready subgroup. "^^ The bottom pane! 
includes those who would be defined as not job-ready using either set of criteria. This group is 
referred to in this analysis as the "not job-ready subgroup." The discussion first compares the job- 
ready and not job-ready subgroups and then examines the modified job-ready subgroup. 

As shown in Table 6.7, participation rates for title not job-ready subgroup (both program and 
control groups) were lower than tihose for the job-ready subgroup. In addition. Project Independence 
had a much larger impact on participation in independent job search and job club for the job-ready 
subgroup: 27.1 percentage points compared to 15.8 percentage points for title not job-ready subgroup. 
This is largely because the not job-ready program group members were much less likely to have 
participated in independent job search or job club activities titian tiheir counterparts in the job-ready 
subgroup. Not surprisingly, tihe program had a larger impact on participation in ABE or GED classes 
for the not job-ready subgroup (9.1 percentage points compared to 2.1 percentage points for the job- 



^Under the modified criteria, job-readiness is defined as having a high school diploma or GED or having 
worked in at least 12 of the 24 months prior to orientation. "Not job-ready" is defined as having no high 
school diploma or GED and not having worked in at least 12 of the 24 months prior to orientation. This 
change went into effect in October 1991, and is not likely to have affected members of titie research sample 
during titi^* follow-up period, since almost all of the program group members who attended orientation did so 
before the change was made. 

^To ascertain v/hich sample members met (or did not meet) tibiese criteria, the program group members' 
Background Information Forms (BIFs) were analyzed with respect to educational attainment levels and work 
experience. That information served as the basis for constructing the job-ready and not job-ready subgroups 
used in the present analysis (and in Table 6.11). However, case managers were allowed to make some 
exceptions to the job-readiness criteria when referring individuals to their first activity. 

^^It was not possible to define research sample members according to the complete set of revised job- 
readiness criteria because the BIFs collected data only on employment in the previous 36 montiiis (in 
accordance with tihe original job-readiness criteria), not the previous 24 montitis (as prescribed in the revised 
job-readiness definition). 
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TABLE 6.7 



IV/O-YEAR IMPACTS OF PROJECT INDEPENDENCE 
ON RATES OF PARTICIPATION IN EMPLOYMENT- RELATED ACTIVITIES. 
BY JOB-READINESS STATUS 





Pmrtrann 
1 I di 1 1 


v.^ui III yJl 
















Subgroup and Participation Measure 


(%) 


(%) 


Differenco 


Change 


Job — readv fhiah ^school diDloma nr fiFH^ 










Pvpr nartiplDatprl in anv pmnln\/mpnt — 

L_Vd |«/Cll llVylf./C(Lwwi III ally CI I lljl^y l M^l IL 










related actK/itv lh\ 


7 


An Q 




OO,** /o 


Fvpr nartirinatpd in indpopndpnt 










ioh cparph nr inh pliih 

j\JiJ dCCllVyll \Jl \\JU V^IUL^ \*^/ 




1 o.o 




i >l Q i O/ 
1 40. 1 TO 


Independent job search 


43.5 


17.5 


26.0 


148.1% 


Job club 


14.1 


o. ( 


8.4 


148.1 % 


Ever participated In education or training (b) 


42.2 


31 .5 


in 7 


*XA 


ABE or GEO 


7.4 


S.4 


2.1 


38.1% 


ESL 


4.8 


2.7 


2.1 


77,3% 


Vocational training or post -secondary education 


32.9 




7 R 


OU.^ /o 


On-the-job training 


4.5 


o.o 


1 \J 


OQ OO/ 


Sample size (total = 749) 


377 


372 






Modified iob-readv (no hiqh school diploma or GED) (c) 








Ever participated in any employment- 










related activity (b) 


66.2 


Of .o 


OR Q 


77 CO/, 
/ f .O /o 


Ever participated in independent 










job search or job club (b) 


38.2 


1 O.w 


1 Q R 


1 HR 09L 


Independent job search 


36.7 


17.9 


18,8 


105.2% 


Job club 


11.9 


5.8 


6.1 


105.2% 


Ever participated in education or training (b) 


48,5 


32.2 


1 R 

1 CO 


RH 7% 


ABE or GED 


32.4 


20.3 


12.0 


59.0% 


ESL 


5.9 


1.7 


4.2 


247.9% 


Vocational training or post -secondary education 


26.5 


13.6 


1 2.9 


95.2% 


On-the-job training 


4,4 


*\ A 


1 ,u 


OU. 1 /O 


Sample size (total = 1 27) 


68 


59 






Not job— ready 










Ever participated in any employment- 










related activity (b) 


55,3 


O 1 .V 


OA A 


7R 79L 
1 O.I /o 


Ever participated In independent 










job search or job club (b) 


277 


11 .9 


15.8 


132.4% 


Independent job search 


26,6 


11.4 


15.1 


132,4% 


Job club 


8.6 


3.7 


4.9 


132.4% 


Ever participated In education or training (b) 


34.0 


23.8 


10,2 


43,0% 


ABE or GED 


23.4 


14.3 


9,1 


63,8% 


ESL 


2.1 


0,0 


2,1 


n/a 


Vocational training or post -secondary education 


10,6 


7.1 


3,5 


49.0% 


On-the-job training 


2.1 


2,4 


-0.3 


-10.5% 


Sample size (total = 89) 


47 


42 







(continued) 
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TABLE 6.7 (continued) 



SOURCE: The 24 -month survey of a subsample of program and control group members. 

NOTES: Participation includes Project Independence and non -Project Independence activities. 

Tests of statistical significance between research groups were not performed. 

Rounding may cause slight discrepancies in calculating sums, averages, and differences. 

Sample members are defined as "job-ready" if they had completed at least 10th grade or were employed 
for at least 1 2 of the 36 months prior to random assignment. They are defined as "not job-ready" if they did not meet both 
criteria. These definitions are based on those used by Project Independence during the the random assignment period. 

"N/a" (not available) is used where the percentage change cannot be calculated. 

(a) Sample members in this category are defined as job-ready under the definition stated above and had 
earned a high school diploma or GED by the time of random assignment. 

(b) Individuals could participate in more than one activity during the follow-up period; therefore, the sum of 
percentages in specific activities may exceed the category percentage. 

(c) Sample members in this category are defined as job-ready under the definition stated above and had 
not earned a high school diploma or GED by the time of random assignment. 
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ready subgroup). On the other hand, the impact on vocational training or post-secondary education 
was larger for the job-ready (7.6 percentage points versus 3.5 percentage points for the not job-ready). 

Although not shown in Table 6.7, the job-ready subgroup was much more likely to have 
participated in both independent job search or job club and education or training as a result of Project 
Independence than the not job-ready subgroup. In particular. Project Independence more than doubled 
the percentage of job-ready program group members who received both types of services, while 
increasing the rate at which not job-ready program group members participated in both types of 
activities by less than one-third. 

As shown in Table 6,8, patterns of impacts on earnings and AFDC payments were different 
for the job- ready and not job-ready subgroups, The job-ready subgroup showed earnings impacts 
in year 1 but a much smaller impact in year 2, AFDC impacts were relatively large in both years, 
although there was some decline over time. Total AFDC reductions exceeded earnings gains, and the 
divergence is likely to increase with more follow-up. In contrast, the not job-ready subgroup showed 
no earnings impact in year 1. The gains in year 2 were relatively large, however: nearly $300 per 
sample member. AFDC savings, on the other hand, were less than $100 in year 1 and were even 
smaller in year 2. The two-year AFDC savings of $144 over two years were less than half the two- 
year earnings gain of $318 and were also less than half the two-year AFDC savings for those in the 
job-ready category. 

An examination of control group outcomes for these two subgroups suggests a possible 
explanation for the opposite patterns of impacts. Earnings for the job-ready subgroup grew over time, 
from $2,809 in year 1 to $3,729 in year 2, Evidently, many control group members in that subgroup 
found employment on their own. Project Independence speeded up job-fmding and got some people 
off AFDC sooner, thereby producing initial earnings gains and AFDC reductions. But, by year 2, 
control group members' earnings had substantially caught up with those of program group members. 
AFDC impacts therefore persisted longer than earnings impacts but may be expected to decrease 
further after year 2. 

The not job-ready subgroup had much less chance of becoming employed on their own. As 
shown in Table 6.8, control group members in this subgroup had year 1 earnings of only $1,008, and 
these increased only slightly to $1,109, in year 2, The two-year average was only about one-third the 
two-year average for the job-ready subgroup. Efforts by Project Independence to increase the skills 
levels of the not job-ready subgroup did, apparently, increase earning power, but the human-capital- 
building activities took much longer to complete than the primarily independent job search activities 
of the job-ready subgrc^up. Earnings impacts for the not job-ready subgroup therefore took longer to 
appear. These earnings impacts may last longer, beyond year 2, because an increase in earning power 
should give program graduates a more permanent labor market advantage than faster job-fmding alone. 
The trade-off appears to have been fewer AFDC case closures. The absence of a strong initial push 
into employment, coupled with the more lengthy program activities, may have been the reason for the 
absence of AFDC impacts in years 1 and 2, Since control group AFDC receipt is continuing to 
decrease among the not job-ready, it is by no means certain that AFDC reductions will begin to appear 
for the not job-ready after year 2, even if their earnings impacts continue. 



Table 6.11 shows impacts for the first two job-readiness categories combined. 
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TABLE 6.8 



TWO-YEAR IMPACTS OF PROJECT INDEPENDENCE 
ON EARNINGS AND AFDC PAYMENTS. 
BY JOB-READINESS STATUS 



Subgroup, Outcome, and Follow-up Period 



Job-ready (high school diploma or GED) (a) 

Average earnings 
Years 1-2 

Yean 
Year 2 

Last quarter of year 2 



Program 
Group 

^ 



6852 

3044 
3803 

1003 



Control 
Group 

($L 



6538 

2809 
3729 

1013 



Difference 



314 * 

235 *** 

80 

-10 



Percentage 
Change 



4.8% 

8.4% 
2.1% 

-1 .0% 



Average AFDC payments received 
Years 1 -2 

Yean 
Year 2 

Last quarter of year 2 



3752 

2057 
1695 

375 



4101 

2259 
1842 

404 



-349 *** 

-202 *** 
-147 *** 

-29 *** 



-8.5% 

-8.9% 
-8.0% 

-7.2% 



Sample size (total = 1 2.537) 



9,356 



3.181 



Modified job— ready 

(no high school diploma or G ED) (b) 

Average earnings 
Years 1 -2 

Yean 
Year 2 

Last quarter of year 2 



3113 

1342 
1772 

458 



2904 

1320 
1584 

438 



209 

22 
187 

20 



7.2% 

1 7% 
1 1 .8% 

4.6% 



Average AFDC payments received 
Years 1 -2 

Year 1 
Year 2 

Last quarter of year 2 



Sample size (total = 2.395) 



4822 

2578 
2244 

509 



4990 

2607 
2382 

542 



-168 

-29 
-139 

-34 



-3.4% 

-1.1% 
-5,8% 

-6.2% 



1.743 



652 



Not job -ready 

Average earnings 
Years 1 -2 

Year 1 
Year 2 

Last quarter of year 2 

Average AFDC payments received 
Years 1 -2 

Year 1 
Year 2 

Last quarter of year 2 



2435 

1035 
1400 

391 



4518 

2467 
2051 

456 



2117 

1008 
1109 

270 



4662 

2564 
2098 

479 



318 

27 

290 

120 



-144 

-97 
-47 

-23 



15.0% 

2.7% 
26,2% 

44.6% 



-3.1% 

-3,8% 
-2,2% 

-4.8% 



Sample s ize (tota[ = 1 .61 6) 



1.349 



467 
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TABLE 6.8 (continued) 



SOURCES: MDRC calculations from Florida Unemployment Insurance (Ul) earnings records and AFDC records. 

NOTES: See Table 6.2. 

Variation in two-year impacts on earnings and AFDC payments across job-readiness subgroups was not 
statistically significant at the 10 percent level. 

Sample members are defined as "job-ready" if they had completed at least 1 0th grade or were employed 
for at least 1 2 of the 36 months prior to random assignment. They are defined as "not job-ready" if they did not meet both 
criteria. These definitions are based on those used by Project Independence during the the random assignment period. 

(a) Sample members in this category are defined as job-ready under the definition stated above and had 
earned a high school diploma or GED by the time of random assignment. 

(b) Sample members in this category are defined as job-ready under the definition stated above and had 
not earned a high school diploma or GED by the time of random assignment. 



-137- 



What were Project Independence's effects on the middle subgroup? In general. Table 6.7 
indicates that the program" s impact on independent job search and job club for the modified job- ready 
subgroup fell in between the comparable impacts for the job-ready and not job-ready subgroups. At 
the same time, the impact on participation in education and training for the modified job-ready 
subgroup was larger than it was for either the job-ready or the not job-ready subgroup. The largest 
impact occurred on participation in vocational training and post-secondary education. 

The estimates in Table 6.8 indicate that there were no statistically significant impacts on two- 
year earnings for the modified job-ready subgroup. The two-year impact on earnings for this subgroup 
was $209 (compared to $314 for the remainder of the job-ready subgroup and $318 for the not job- 
ready subgroup). The two-year AFDC savings were $168 - similar in magnitude to the savings for 
the not job-ready ($144), but less than the savings for the job-ready ($349). The overall differences 
in impacts among the three subgroups were not statistically significant. Nevertheless, the increase in 
impacts on earnings and AFDC payments for the middle group from year 1 to year 2 is consistent with 
the relatively large impact on education and training participation rates reported for that group. It also 
suggests that the up-front independent job search did not have a payoff for this group. 

B* Impacts on Long-Term AFDC Recipients 

This section addresses the question of whether Project Independence's impacts varied for 
subgroups defined on the basis of their prior AFDC history. This issue has taken on particular 
importance because the Family Support Act of 1988 requires states to focus more than half of their 
resources on target groups that include longer-term AFDC recipients. For purposes of the present 
analysis, prior welfare history can be analyzed along two dimensions. The first is the number of years 
of prior AFDC receipt. The second is whether the sample member was receiving AFDC at the time 
of random assignment. Sample members who were applying for AFDC at the time of random 
assignment are called "applicants"; those who were already on AFDC are called "recipients." Both 
applicants and recipients may have had short or long histories of prior AFDC receipt, possibly in one 
or more prior spells of receipt. 

Several previous evaluations of welfare-to-work programs have examined impacts for sub- 
groups defined by prior AFDC receipt patterns. These evaluations have found that applicants with no 
prior AFDC receipt have generally produced below-average earnings gains and AFDC reductions. 
This is because many first-time applicants find jobs and leave AFDC fairly quickly even without 
program assistance. The ability of a program to increase job-finding and welfare case closure, above 
and beyond normal behavior, is therefore limited for this subgroup. The impacts for those with some 
previous AFDC receipt have varied, depending on the program and the outcome measure. Long-term 
AFDC recipients have usually accounted for a major share of AFDC reductions (in programs that had 
any welfare impacts), but have not as consistently obtained sizable earnings gains. In the absence of 



^^Specifically, federal law requires that a minimum of 55 percent of JOBS expenditures be directed to 
participants who belonged to specified "target groups." One of these target groups includes those who have 
received AFDC for a minimum of 36 of the previous 60 months. 

^^See Friedlander, 1988; Gueron and Pauly, 1991; and Riccio, Friedlander, and Freedman, 1994. One 
exception was the San Diego Saturation Work Initiative Model (SWIM) Program, which generated substantial 
impacts for first-time applicants. For those results, see Friedlander and Hamilton, 1993. 
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a program intervention, long-term AFDC recipients often tend to remain on AFDC for several years 
into the future, so any program-induced changes in their behavior can result in relatively large AFDC 
reductions. However, the very low skills levels of many long-termers reduce their ability to sustain 
employment over the long run. These low levels of skills may partly explain the lack of consistent 
earnings impacts for this subgroup in the low- to moderate-cost programs of the past. 

The Project Independence research sample was divided into three subgroups based on sample 
members' history of AFDC receipt prior to random assignment: first-time applicants, AFDC receipt 
totaling less than two years on the person's own or spouse's case ("short-term" applicants and 
recipients), and AFDC receipt totaling two years or more on the person's own or spouse's case ("long- 
tenn" applicants and recipients).^"* Note that "applicants" to AFDC often have received AFDC 
previously, and such persons may be classified as short term or long term. As shown in Table 6.9, 
the overall participation rates among program group members and the program's impact on 
participation rates for the three subgroups were generally similar. The primary exceptions were the 
relatively high rate of participation in ESL classes among first-time applicants (in both the program 
and control groups) and the relatively low rate of participation in vocational training or post-secondary 
education classes among long-term applicants and recipients in the program group. 

Impact estimates for the welfare history subgroups are displayed in Table 6.10. As has been 
the pattern in previous studies, first-time applicants realized a small and not statistically significant 
earnings impact: $54 over the two-year follow-up. Surprisingly, those who had two years or more 
of prior AFDC receipt generated the largest two-year earnings impact, $738 per sample member. The 
$87 earnings impact for those with less than two years of prior AFDC receipt was similar to the 
impact for first-time applicants. The differences in the two-year earnings impacts among these three 
subgroups were statistically significant. Despite their larger impacts, program group members with 
two years or more of previous AFDC receipt had average earnings that were substantially lower than 
the earnings of those with fewer months of past AFDC receipt. Even in year 2, the average earnings 
of program group members with longer prior AFDC receipt were $2,413, compared to $3,134 for 
those with less than two years of AFDC receipt and $3,775 for first-time applicants (zero earnings 
were included in the averages for sample members who were not employed). 

AFDC payments impacts for the three subgroups were closer in size than the earnings impacts 
and were not correlated with subgroup earnings impacts. The two-year AFDC savings were $407 for 
those with two years or more of receipt, $295 for those with less than two years of receipt, and $252 
for first-time applicants. The differences among the three subgroups in these impacts on AFDC 
payments were not statistically significant. 

Each of the two subgroups with prior AFDC receipt can be further divided by their status as 
applicants or recipients. Specifically, there are applicants and recipients with less than two years of 
prior AFDC receipt and applicants and recipients with two years or more prior AFDC receipt. 
Combined with the first-time applicants, this leads to a five-way breakdown of the sample. The 
impacts for each of these five subgroups are reported in Table 6.11. This table reveals that applicants 



'"^These definitions differ from those used in other MDRC studies of welfare-to-work programs. In most 
of these studies, applicants were defined as a single group, and only ongoing recipients were divided into 
subgroups defined by long-term and shon-term AFDC histories. 
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TABLE 6.9 



TWO-YEAR IMPACTS OF PROJECT INDEPENDENCE 
ON RATES OF PARTICIPATION IN EMPLOYMENT- RELATED ACTIVITIES, 

BY AFDC HISTORY 





Program 


Control 








Group 


Group 




Percentage 


Subgroup and Participation Measure 


{%) 


(%) 


Difference 


Change 












Ever participated in any employment- 










related activity (a) 


63.3 


39.0 


24.3 


62.3% 


Ever participated in independent 










job search or job club (a) 


41.2 


13.2 


28.0 


212.4% 


Independent job search 


39.6 


12.7 


26.9 


212.4% 


Job club 


12.8 


4.1 


8.7 


212.4% 


Ever participated in education or training (a) 


43.7 


33.5 


10.2 


30.4% 


ABE or GED 


10.1 


5.5 


4.6 


83.1% 




8.0 


6.6 


1 .9 


29.6% 


Vocational training or post —secondary education 


30.2 


22.0 


8.2 


37.2% 


On— the— job training 


4.0 


2.8 


1 .3 


46.2% 


Sample size (total = 381) 


199 


182 
















Ever participated in any employment- 










related activity (a) 


64.9 


42.9 


22.0 


51.3% 


Ever participated in independent 










job search or job club (a) 


43.6 


20.7 


22.9 


110.7% 


Independent job search 


41.9 


19.9 


22.0 


110.7% 


Job club 


13.5 


6.4 


7.1 


110.7% 


Ever participated in education or training (a) 


40.6 


30.0 


10.6 


35.4% 


ABE or GED 


12.7 


9.3 


3.4 


37.0% 






0.0 




n/a 


Vocational training or post— secondary ed'jcation 


30.9 


21 .4 


9.5 


44.2% 


On— the —job training 


3.0 


5.0 


— 2.0 


—39.4% 


Sample size (total = 305) 


165 


140 
















Ever participated in any employment - 










related activity (a) 


61.8 


39.3 


22.5 


57.4% 


Ever participated in independent 










job search or job club (a) 


43.1 


21 .5 


21.6 


100.7% 


Independent job search 


41.4 


20.6 


20.8 


100.7% 


Job club 


13.4 


6.7 


6.7 


100.7% 


Ever participated in education or training (a) 


39.0 


28.2 


10.8 


38.3% 


ABE or GED 


13.8 


9.8 


4.0 


40.7% 


ESL 


1.6 


1.2 


0.4 


32.5% 


Vocational training or post-secondary education 


25.2 


20.9 


4.3 


20.8% 


On-the-job training 


5.7 


2.5 


3.2 


132.2% 


Sample size (total = 286) 


123 


163 







SOURCE: The 24-month survey of a subsample of program and control group members. 



NOTES: Participation includes Project Independence and non -Project Independence activities. 
Tests of statistical significance between research groups were not performed. 
Rounding may cause slight discrepancies in calculating sums, averages, and differences. 
"N/a" (not applicable) is used where the percentage change cannot be calculated. 

(a) Individuals could participate in more than one activity during the follow-up period; therefore, the sum of 
percentages in specific activities may exceed the category percentage. 

(b) This subgroup includes both applicants and recipients, and refers to the total number of months accumulated 
from one or more spells on an individual's or spouse's AFDC case. It does not include AFDC receipt under a parent's name. 
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TABLE 6.10 



TWO-YEAR IMPACTS OF PROJECT INDEPENDENCE 
ON EARNINGS AND AFDC PAYMENTS. 
BY AFDC HISTORY 



Subgroup, Outcome, and Follow-up Period 



Program 
Group 

($L 



Control 
Group 



Difference 



Percentage 
Change 



First-time applicant 

Average earnings 
Years 1-2 

Year 1 
Year 2 

Last quarter of year 2 

Average AFDC payments received 
Years 1 -2 

Yean 
Year 2 

Last quarter of year 2 



Sample size (total = 7,118) 



6734 

2959 
3775 

1004 



3286 

1830 
1456 

313 



6680 

2920 
3759 

992 



3538 

1972 
1566 

351 



54 

39 
15 

12 

-252 *** 

-141 *** 
-110 *** 

-.37 *** 



0.8% 

1 .3% 
0.4% 

1 .2% 



-7.1% 

-7.2% 
-7.1% 

-10.6% 



Less than two years (a) 

Average earnings 
Years 1-2 

Yean 
Year 2 

Last quarter of year 2 

Average AFDC payments received 
Years 1 -2 

Yean 
Year 2 

Last quarter of year 2 



Sample size (total = 5,163) 



5,311 



1.807 



5651 

2516 
3134 

827 



4188 

2274 
1914 

426 



5564 

2367 
3197 

872 



4484 

2474 
2010 

439 



87 

150 
-63 

-45 



-295 *** 

-200 *** 
-96 * 

-13 



1 .6% 

6.3% 
-2.0% 

-5.1% 



-6.6% 

-8.1% 
-4.8% 

-2.9% 



Two years or mo re (a) 

Average earnings 
Years 1-2 

Year 1 
Year 2 

Last quarter of year 2 

Average AFDC payments received 
Years 1 -2 

Year 1 
Year 2 

Last quarter of year 2 
Sample size (total = 4,620) 



3.851 



1.312 



4382 

1969 
2413 

616 



4946 

2642 
2304 

529 



3644 

1584 
2060 

584 



5353 

2831 
2522 

566 



3.373 



1.247 



738 *** 

385 *** 
353 *** 

32 



-407 *** 

-189 *** 
-218 *** 

-37 ** 



20.3% 

24.3% 
17.1% 

5.5% 



-7.6% 

-6.7% 
-8.6% 

-6.6% 



SOURCES: MDRC calculations from Florida Unemployment Insurance (Ul) earnings records and AFDC records. 
NOTES: See Table 6.2. 

Variation in two-year impacts on earnings across welfare history categories was statistically significant at the 
5 percent level. Variation in two-year Impacts on AFDC payments across welfare history categories was not statistically 
significant at the 10 percent level. 

(a) This subgroup Includes both applicants and recipients, and refers to the total number of months 
accumulated from one or more spells on an Individual's or spouse's .AFDC case. It does not Include AFDC receipt under 
a parent's name. 
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TABLE 6.11 

TWO-YEAR IMPACTS OF PROJECT INDEPENDENCE ON EARNINGS AND AFDC PAYMENTS FOR ADDITIONAL SUBGROUPS 



Average Total E arnings 



Average Total AFDC Payments 



Subgroup and Follow-up Period 



Program 
Gr oup ($) 



Control 
Group ($) 



Difference 



Percentage 
Change 



Program 
Group ($) 



Control 
Group ($) 



Difference 



Job ~/ead iness status _{a) 
Job ready 
Years'V-2 

Year 1 
Year 2 

Not job -ready 
Years 1-2 

Year 1 
Year 2 

AFDC history 

First -time applicant 
Years 1 -2 

Year 1 
Year 2 

Applicant, lessjhan 2 years 
Years 1-2 

Year 1 
Year 2 

Applicant, 2 years or more 
Years 1 -2 

Year 1 
Year 2 

Recipient, less than 2 years 
Years 1-2 

Year 1 
Year 2 

Recipient, 2 years or more 
Years 1-2 

Year 1 
Year 2 



(sample size = 14,932) 



Percentage 
Change 



6250 


5963 


287 * 


4.8% 


3924 


4241 


-317 


-7.5% 


0770 




•inc AAA 


7 CO/ 
/ .070 




OQi il 


«- i 7*5 
— I/O 


-«7 ROA 


3480 


3388 


92 


2.7% 


1783 


1927 


— •144 


-7.5% 


= 1,816) 


















P1 1 7 

^ 1 1 r 


316 


1 «J . u /o 


451 8 


4662 


-144 


-3.1% 


1035 


1008 


27 


2.7% 


2467 


2564 


-97 


-3.8% 


i /inn 


\ i no 




OR OOA 

cJQ /o 


C\i^ 1 


<cuyo 




—9 ooA 
/o 


= 7,118) 
















6734 


6680 


54 


0.8% 


3286 


3538 


-252 


-7.1% 


2959 


2920 


39 


1 .3% 


1830 


1972 


^\^\ AAA 


-7,2% 


3775 


3759 


15 


0.4% 


1456 


1566 


-110 


-7.1% 


= 4.373) 
















5891 


5678 


213 


3.8% 


4055 


4333 


-278 *** 


-6.4% 


2638 


2405 


232 * 


9.7% 


2197 


2395 


-198 


-8.3% 


3254 


3273 


-19 


-0.6% 


1858 


1938 


-60 


-4.1% 


= 3,591) 
















4791 


3969 


822 *** 


20.7% 


4501 


4951 


-451 


-9.1% 


2173 


1743 


430 *** 


24.7% 


2430 


2619 


-189 


-7,2% 


2618 


2226 


392 ** 


17.6% 


2070 


2332 


-262 


-11.2% 


= 790) 
















4402 


4629 


-227 


-4.9% 


4904 


5356 


-452 ** 


-8.4% 


1885 


1990 


-105 


-5.3% 


2688 


2938 


-249 *** 


-8.5% 


2516 


2639 


-123 


-4.7% 


2216 


2419 


-203 


-8,4% 


= 1 ,029) 
















2956 


2440 


516 


21.1% 


6511 


6768 


-257 * 


-3.8% 


1267 


969 


298 * 


30.7% 


3386 


3565 


-,179 AAA 


-5,0% 


1689 


1471 


218 


14.8% 


3125 


3204 


-78 


-2.4% 
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TABLE 6.11 (continued) 







Average Total Earnings 






Program 


oontroi 




Percentage 


^yba^pup and FolJow-upJ^e Group ($) 




Difference 


Change 


Level of disadvantaqe 










f^'jst --timeappJicant 


(sample size = 7,118) 








Years 1—2 


6734 


6680 


54 


0.8% 


Year 1 


2959 


2920 


39 


1 .3% 


Year 2 


3775 


3759 


15 


0.4% 


Returning applicant 


(sample size = 7,964) 








Years 1 -2 


5391 


4917 


474 ** 


9.6% 


Tear i 


2426 


2113 


313 *** 


14.8% 


Year 2 


2965 


2804 


161 


5.7% 


Less disadvantaged recipient 


(sample size = 1 ,442) 








Years 1 -2 


4239 


3926 


314 


8.0% 


Year 1 


1824 


1648 


176 


10.7% 


Year 2 


2415 


2278 


137 


6.0% 


More disadvantaged recipient 


(sample size = 377) 








Years 1-2 


1219 


C\^ A 

y 1 4 


304 


33.3% 


Year 1 


488 


318 


170 


53.5% 


Year 2 


731 


597 




22.5% 


Employed in the year prior to 








random assignment 










Yes 


(sample size = 11.146) 








Years 1 -2 


7279 


7103 


176 


2.5% 


Year 1 


3246 


3120 


126 


4.0% 


Year 2 


4033 


3983 


50 


1 .3% 


No 


(sample size = 7,087) 








Years 1 -2 


3390 


3071 


319 * 


10.4% 


Year 1 


1450 


1268 


181 ** 


14.3% 


Year 2 


1940 


1803 


137 


7.6% 
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Average Total AFDC Payments 



Program Control Percentage 
Gro up ($) Group ($) Difference Change 



3286 3538 -252 *** -7.1% 

1830 1972 -141 *** -7.2% 

1456 1566 -110 *** -7.1% 



4259 4605 -346 *** -7.5% 

2303 2494 -191 *** -7.7% 

1956 2110 -155 *** -7.3% 



5493 5891 -398 *** -6.6% 

2946 3186 -240 *** -7.5% 

2547 2706 -159 * -5.9% 



7017 7207 

3604 3710 
3413 3497 



-189 -2.6% 

-105 -2.8% 
-84 -2.4% 



3885 4168 -283 *** -6.8% 

2099 2262 -163 *** -7.2% 

1786 1906 -120 *** -6.3% 



4253 4489 

2349 2484 
1904 2004 



-236 *** -5.2% 

-136 *** -5.5% 
-100 ** -5.0% 

(cor>tinued) 
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TABLE 6.11 (continued) 







Average Total Earnings 






Average Total AFDC Payments 




Subgroup and Fpllow 


n I vJ^i cu 1 1 

-up Period Group ($) 


Control 
Group ($) 


Difference 


Change 


PrnnrflfYi 
1 1 ^J^i cu 1 1 

Group ($) 


Control 
Group ($) 


Difference 


Change 


cinriiciiy 

W!l'lei..npn - Hispa nic 
Years l -2 


(sample size = 6,1 1?) 

5867 


5642 


225 


4.0% 


3206 


3534 


-329 *** 


-9.3% 


Year 1 
Year 2 


2682 
3185 


2524 
3118 


i58 
67 


6.3% 
2,1% 


1826 
1379 


2019 
1515 


-193 *** 
-136 *** 


-9.6% 
-9.0% 


8lack, non-Hispanic 
Years 1 -2 


(sample size = 6,875) 
5943 


5877 


66 


1.1% 


4779 


5086 


-307 *** 


-6.0% 


Year 1 

Year 2 


2569 
3374 


2526 
3351 


43 
23 


1 .7% 
0,7% 


2526 
2254 


2685 
2401 


-159 *** 
-148 *** 


-5.9% 
-6.2% 


Hispanic 
Years 1 -2 


(sample size = 4,044) 
5168 


4695 


474 * 


10.1% 


4064 


4236 


-172 * 


-4.1% 


Year 1 
Year 2 


2270 
2899 


1973 
2722 


297 ** 
177 


15.0% 
6.5% 


2226 
1838 


2344 
1892 


-118 ** 
-54 


-5.0% 
-2.9% 


Acisin anH r^thor 
r\oic«ll oil lU Lili ici 

Years 1 -2 


7140 


3869 


3271 u** 


84.6% 


3423 


4042 


-618 u 


-15.3% 


T KSai \ 

Year 2 


4446 


1468 
2401 


1226 * 
2045 ** 


85.2% 


1990 
1433 


2375 
1666 


-385 
-233 


— 1R P% 

-14.0% 


PrifTtrin/ Irinnimnp 
~iiiiiciiy iciii^uavjc 

English 
Years 1 -2 


(sample size = 13,883) 
6042 


5772 


270 * 


4,7% 


3990 


431 1 


-321 *** 


-7.4% 


Year i 
Year 2 


2676 
3364 


2522 
3250 


156 ** 
114 


6.2% 
3,5% 


2174 
1816 


2349 
1962 


-175 *** 
—146 *** 


-7.5% 
-7.4% 


Spanish 
Years 1 -2 


(sample size 3,232) 
4552 


4371 


181 


4,1% 


4194 


4280 


-85 


-2.0% 


Year 1 
Year 2 


1994 
2556 


1816 
2554 


177 
4 


9,8% 
0,2% 


2288 
1906 


2376 
1904 


-87 * 

2 


-3.7% 
0.1% 


Other 
Years 1-2 


(sample size = 303) 
5521 


4539 


982 


21 .6% 


4310 


4849 


-540 


-11.1% 


Year 1 
Year 2 


2274 
3246 


2102 
2437 


172 
809 


8,2% 
33.2% 


2342 
1968 


2562 
2288 


-220 
-320 


•-8.6% 
-14.0% 
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TABLE 6.11 



(continued) 



Average Total Earnings Avera ge Total AFDC Payments 

Program Control Percentage Program Control Percentage 
Subgroup and Follow-up Period Group ($) Group ($) Dif ference Change Group ($) Group ($) Difference Change 

Participating in an educ ation 
QL if aJr i 09. pr ogram at th e 
t]me of random assignment 

Yes (sample size - 2,061) 



Years 1 -2 


6706 


5765 




1 o.O /o 






-~ODO 


— o.o% 


Year 1 


2663 




A7A ** 






2413 


— 211 * 


—8.8% 


Year 2 


3843 


3376 


466 * 


13.8% 


1803 


1955 


-152 ** 


-7.8% 


No 


(sample size = 15,025) 
















Years 1 -2 


5602 


5421 


181 


o.o /o 


HUOO 


zi9Qn 




C AO/ 

— D.U% 


Year 1 


2495 




1 O 1 


O.Dyo 






— ^ /yj * ** 


-6.3% 


Year 2 


3107 


3058 


50 


1 ,6% 


1838 


1947 


-110 *** 


-5.6% 


Number of children at the time 
















of random assignment (b) 


















One * " 


(sample size = 7,858) 
















Years 1 -2 


5780 


5565 


216 


3.9% 


3232 


3471 


-239 *** 


-6.9% 


Year 1 


2567 


2436 


131 


5.4% 


1755 


1894 


-139 *** 


-7.4% 


Year 2 


3214 


3129 


85 


2.7% 


1477 


1577 


-99 *** 


-6.3% 


Two 


(sample size = 5.575) 
















Years 1 -2 


6102 


5759 


343 


6.0% 


4116 


4512 


-397 *** 


-8.8% 


Year 1 


2716 


2505 


210 * 


8.4% 


2267 


2494 


-226 *** 


-9.1% 


Year 2 


3386 


3253 


133 


4.1% 


1848 


2019 


-170 *** 


-8.4% 


Three or more 


(sample size = 3,795) 
















Years 1 -2 


5255 


4981 


274 


5.5% 


5642 


5802 


-159 


-2.7% 


Year 1 


2289 


2085 


204 


9.8% 


3067 


3138 


-71 


-2.3% 


Year 2 


2966 


2896 


70 


2.4% 


2575 


2664 


-89 


-3.3% 



(continued) 
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TABLE 6.11 (continued) 



SOURCES: MDRC calculations from Florida Unemployment insurance (Ui) earnings records and AFDC records. 
NOTES: See Table 6.2. 

The symbol "u" indicates that, because of very small sample sizes, the Impact estimate shown is unreliable; asterisks following the symbol 
indicate that there was a statistically significant effect, though its magnitude could not be reliably measured. 

(a) Sample members are defined as "job-ready" if they had completed at least 10th grade or were employed for at least 1 2 of the 36 months prior to random 
assignment. They are defined as "not job~ready" if they did not meet both criteria. These definitions are based on those used by Project Independence during the 
random assignment period. 

(b) This section excludes sample members who reported having no children. 

Variation in two-year impacts across subgroups was statistically significant at the 10 percent level or greater as follows: 



Subgroup 


Earnings 


AFDC Pavments 


Job-readiness status 


No 


No 


AFDC history 


No 


No 


Level of disadvantage 


No 


No 


Employed in the year prior to 






random assignment 


No 


No 


Ethnicity 


No 


No 


Prmary language 


No 


Yes 


Participating in an education or training 






program at the time of random assignment 


No 


No 


Number of children at time of 






random assignment 


No 


No 
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with more than two years of prior AFDC receipt tended to experience large and sustained earnings 
impacts. In other words, earnings impacts were largest for a subgroup in the middle of the 
spectrum of prior welfare receipt, neither among those with the least prior welfare receipt nor among 
those with the longest prior welfare histories. This finding is consistent with earlier research. 
AFDC impacts were smallest for first-time applicants and recipients with a welfare history of two 
years or more. 



V. Impacts for Other Subgroups 

Table 6.11 presents impacts for a number of other subgroups of the Project Independence 
research sample, and a summary of the findings is presented in this section. Two-year impacts on 
AFDC payments were similar for sample members with and without employment in the year prior to 
random assignment; earnings impacts were slightly higher for those who were not employed in the year 
prior to random assignment. 

Across ethnic subgroups, earnings impacts were large for the Hispanic and Asian/other 
categories. This latter subgroup was quite small, however, and impact estimates for it are urneliable 
as a consequence. Two-year earnings impacts for Hispanics were $474, although the effect appeared 
to be decreasing over time. For whites and blacks, earnings impacts were below the full-sample 
average. Their AFDC impacts were larger than those for Hispanics, however, and exceeded their 
earnings impacts. Hispanic earnings impacts appeared not to have accrued to sample members who 
were not proficient in English: Two-year impacts on both earnings and AFDC payments were small 
for sample members whose primary language was Spanish. Differences in two-year impacts across 
ethnic subgroups were not statistically significant. Differences in two-year earnings impacts across 
primary language subgroups were not statistically significant, but two-year differences in AFDC 
payments impacts were. 

About one in eight sample members were already participating in an education or training 
program at the time of their referral to Project Independence. For those individuals, it is a question 
whether Project Independence could achieve any impacts above and beyond any impacts that might 
have been achieved by the pre-Project Independence activities. In fact, earnings gains were 
particularly large for this subgroup, and AFDC reductions were above the full-sample average. Cross- 
subgroup impact differences were not statistically significant. 

Across subgroups based on sample members' number of children, differences in impacts were 
not statistically significant for either two-year earnings or two-year AFDC payments. Two-year 
earnings impacts were not widely different. Two-year AFDC reductions were somewhat larger for 
the subgroup with two children and smaller for the subgroup witli three or more children, but the 
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^^Collectively, the differences in first-year earnings impacts among the five subgroups were statistically 
significant. However, the only pair-wise differences in these impacts that were statistically significant were: 
(1) the differences between impacts for applicants with two years or more of prior AFDC receipt and those 
for first-time applicants, and (2) the differences between impacts for applicants with two years or more of 
prior AFDC receipt and those for recipients with less than two years. 

*^See Friedlander, 1988. 



■147- 



2 



differences were not large enough to suggest that Project Independence was more effective with any 
one of these subgroups. 



VL Two- Year Impacts by County 

The impacts on participation in employment-related activities for each county are presented in 
Table 6.12.^^ The table indicates considerable variation across the counties in the mix of services 
utilized and in Project Independence's impact on participation. For example. Project Independence's 
impact on rates of participation in any employment-related activity ranged from a high of 35.7 per- 
centage points for Hillsborough to a low of 11.9 percentage points for Orange. These two counties 
were also at the extremes of impacts on participation in independent job search or job club (31.8 
percentage points for Hillsborough compared to 11.8 percentage points for Orange). Impacts on 
participation in education and training activities were largest for Dade (26.7 percentage points) - with 
an especially large impact on participation in ESL (15.9 percentage points) - and smallest for Orange 
(2.9 percentage points). 

Table 6.13 presents the two-year impacts on earnings and AFDC payments for each of the nine 
research counties and for the full research sample. The impacts on two-year earnings were positive 
for all the counties except Duval and Hillsborough, although they were statistically significant only in 
Lee and Orange. Although not statistically significant, the two-year impacts on earnings exceeded the 
all-county average impact in Broward, Dade, Orange, Pinellas, and Volusia. The table also shows that 
two-year AFDC payments were reduced in all research counties except Bay, and that these reductions 
were statistically significant in Broward, Dade, Duval, Hillsborough, and Orange. Hillsborough and 
Orange achieved the largest AFDC reductions. Earnings gains exceeded AFDC payments reductions 
in all counties except Duval and Hillsborough. 

It is important to note that the differences among counties in two-year impacts were not 
statistically significant. This means that much of the apparent variation in county impact estimates 
displayed in Table 6.13 may well have stemmed from chance rather than from real differences in 
county performance. Thus, the differences among counties displayed in Table 6.13 should not be 
interpreted as a reliable ranking of the relative effectiveness of the county-specific Project 
Independence programs. There may, however, be important reasons why some county impact 
estimates are relatively high and others relatively low. 

This report does not attempt to identify the causes of variation in county impacts. However, 
it is worth noting that there are no obvious or consistent relationships between the level of county- 
specific impacts and county characteristics such as labor market conditions, program service 
availability and quality, or program participation patterns. This can be seen by comparing the county- 
specific impacts in Table 6.13 with the county-specific participation information in Table 6.12, the 
service availability and quality information in Table 3.1, and the labor market information in Table 
1.1. It may be that a combination of these factors, or some other factor altogether, explains the 
variation in county impacts. In general, however, no conclusions can be drawn from the evidence 



^^Because of small sample sizes, special caution should be used in interpreting the county-specific 
participation pattenis and estimates of Project Iniependence's impact on participation, 
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TABLE 6.12 



TWO-YEAR IMPACTS OF PROJECT INDEPENDENCE 
ON RATES OF PARTICIPATION IN EMPLOYMENT-RELATED ACTIVITIES. 

BY COUNTY 



County and Participation Measure 


Program 
Group 

(%) 


Control 
Group 
(%) 


Difference 


Percentage 
Chanae 


Bay 










Ever participated in any employment- 
related activity (a) 


64.7 


44.0 


20.7 


47.1% 


Pvpr rv^rtipirvitpH in inHpn^nHpnt 
iob ^parph or ioh pliih 

Independent job search 
Job club 


39.5 
12.8 


CXJXJ 

19.2 
6.2 


20.3 
6.6 


105.9% 
105.9% 
105.9% 


Ever participated in education or training (a) 
ABE or GED 
ESL 

Vocational training or post -secondary education 
On-the-job training 


47.1 

5.9 
0.0 
41.2 
9.8 


36.0 
14.0 
4.0 
28.0 
10.0 


11.1 
-8.1 
-4.0 
13.2 
-0.2 


30.7% 
-58.0% 
n/a 
47.1% 
-2.0% 


Sample size (total = 101) 


51 


50 






Broward 










Ever participated in any employment- 
related activity (a) 


67.4 


46.3 


21.1 


45.6% 


Evpr Dartif inatpd in inripnpnripnt 
iob ^parph nr ioh pliih 

Independent job search 

Job club 


47.0 
15.2 


oo o 

c.c.,c. 

21.3 
6.9 


25.7 
8.3 


1 20.4% 
1 20.4% 


Ever participated in education or training (a) 
ABE or GED 
ESL 

Vocational training or post— secondary education 
On-the-job training 


42.9 
20.4 

0.0 
36.7 

2.0 


37.0 
11.1 

7.4 
20.4 

1.9 


5.8 
9.3 
-7.4 
16.4 
0.2 


15.7% 
83.7% 
n/a 
80.3% 
10.3% 


Sample size (total = 103) 


49 


54 






Dade 










Ever participated in any employment - 
related activity (a) 


67.0 


35.7 


31.3 


87.7% 


Ever participated in independent 
job search or job club (a) 

Independent job search 

Job club 


47.3 
45.4 
14.6 


19.1 
18.3 
5.9 


28.2 
27.1 
8.7 


148.0% 
148.0% 
148.0% 


Ever participated in education or training (a) 
ABE or GED 
ESL 

Vocational training or post-secondary education 
On -the - job training 


51.7 
9.9 
23.1 
29.7 
2.2 


25.0 
3.6 
7.1 

17.9 
2.4 


26.7 
6.3 
15.9 
11.8 
-0.2 


106.6% 
177.0% 
223.2% 
66.1% 
-7.6% 


Sample size (total = 1 75) 


91 


84 
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TABLE 6.12 (continued) 



County and Participation Measure 


Program 
Group 
(%) 


Control 
Group 
(%) 


Difference 


Percentage 
Change 


Duval 










Ever participated in any ennploynnent- 
related activity (a) 


52.0 


32.1 


19.9 


62.1% 


Ever participated in independent 
job search or job club (a) 

Independent job search 

Job club 


32.0 
307 
9.9 


11.3 
10.9 
3,5 


20.7 
19.9 
6.4 


182.7% 
182.7% 
182.7% 


Ever participated in education or training (a) 
ABEorGED 
ESL 

Vocational training or post -secondary education 
On-the -job training 


36.0 
12.0 

2.0 
24.0 

6.0 


30.2 
9.4 
0.0 

18.9 
1.9 


5.8 
2.6 
2.0 
5.1 
4.1 


19.2% 
27.3% 
n/a 
27.2% 
217.5% 


Sample size (total = 1 03) 


50 


53 






Hillsborouqh 










Ever participated In any employment- 
related activity (a) 


61.1 


25.5 


35.7 


140.1% 


Ever participated in independent 
job search or job club (a) 

Independent job search 

Job club 


46.3 
44.4 
14.4 


14.6 
14.0 
4.5 


31.8 
30.5 
9.6 


218.6% 
218.2% 
218.2% 


Ever participated In education or training (a) 
ABEorGED 
ESL 

Vocational training or post -secondary education 
On-the-job training 


27.8 
11.1 

1.9 
18.5 

1.9 


16.4 
3.6 
1.8 

10.9 
1.8 


11.4 
7.5 
0.0 
7.6 
0,0 


69.8% 
205.2% 

1 .6% 
69.8% 

1.6% 


Sample size (total = 109) 


54 


55 






Lee 










Ever participated in any employment- 
related activity (a) 


64.8 


42.9 


22.0 


51 .2% 


Ever participated in independent 
job search or job club (a) 

Independent job search 

Job club 


44.4 
42.7 
13.8 


16.1 
15.4 

D.O 


28.4 
27,2 
8.8 


176.5% 
176.5% 

4 ~7C CO/ 

1 /D.D% 


Ever participated in education or training (a) 
ABE or GED 
ESL 

Vocational training or post -secondary education 
On-the-job training 


37.0 
11.1 

1.9 
22.2 

5.6 


33.9 

10.7 
1.8 

23.2 
3.6 


3.1 
0.4 
0.1 
-1.0 
2.0 


9.2% 
3-7% 
3.4% 
-4.3% 
55.7% 


Sample size (total = 110) 


54 


56 
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TABLE 6.12 (continued) 





Program 


Control 








Group 


Group 




Percentage 


oouniy ana ramcipaiion ivieasure 


(%) 


(%) 


Uitrerence 


Change 


\jr afiQe 










Ever participated in any employment- . 










reiaieo aciiviiy [3.) 


53.5 


41 .5 


1 1 .9 


28.8% 


Ever participated in independent 










job search or job club (a) 


34.5 


22.6 


1 1.8 


52,3% 


Independent job search 


33.1 


21.7 


11.4 


52.3% 


Job club 


10.7 


7.0 


3.7 


52.3% 


Ever participated in education or training (a) 


29.3 


26.4 


2.9 


10,9% 


ABE or GED 


1 2.1 


9.4 


2.6 


28.0% 


ESL 


0.0 


0.0 


0.0 


n/a 


Vocational training or post -secondary education 


15.5 


15.1 


0.4 


2.8% 


On— the— job training 


3.5 


3.8 


—0.3 


—8.5% 


Sample size (total = 111) 


58 


53 






Pinellas 










Ever participated In any employment - 










related activity (a) 


66.7 


47.2 


19.5 


41 .3% 




Ever participate?.* independent 










job search or jcc club (a) 


38.6 


20 .8 


1 7.9 


86.0% 


Independent job search 


37.1 


19.9 


17.1 


86.0% 


Job club 


12.0 


6.4 


5.5 


86.0% 


Ever participated in education or training (a) 


49.1 


35.9 


13.3 


37.0% 


ABE or GED 


21.1 


7.6 


13.5 


178.8% 


ESL 


0.0 


1.9 


-1.9 


n/a 


Vocational training or post -secondary education 


33.3 


30.2 


3.1 


10.4% 


On —the —job training 


7.0 


1 .9 


5.1 


271 ,4% 


oamjjie size (lOiai — i luj 


Of 


53 






Volusia 










Ever participated In any employment - 










related activity (a) 


75.0 


49.0 


26.0 


53.0% 


. . , 
Ever participated in Independent 










jOD searcn or jod ciud \a) 




£. \ .b 






Independent job search 


46,3 


20.7 


25.6 


123,5% 


Job club 


14.9 


6.7 


8.3 


123,5% 


Ever participated in education or training (a) 


46.4 


37.3 


9.2 


24.6% 


ABE or GED 


7.1 


7.8 


-0.7 


-8,9% 


ESL 


0.0 


0.0 


0.0 


n/a 


Vocational training or post-secondary education 


42.9 


31.4 


11.5 


36.6% 


On-the-job training 


0.0 


2.0 


-2.0 


-100.0% 


Sample size (total = 107) 


56 


51 







SOURCE: The 24-month survey of a subsample of program and control group members. 



NOTES: Participation includes Project Independence and non-Project independence activities. 

Tests of statistical significance between research groups were not performed. 

Rounding may cause slight disrepancles when calculating sums, averages, and differences. 

"N/a" (not applicable) is used when there is zero participation by program or control group members and. 
therefore, a percentage change calculation is meaningless. 

(a) Individuals could participate in more than one activity during the follow-up period; therefore, the sum 
of the percentages in specific activities may exceed the category percentage. 
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TABLE 6.13 

TWO-YEAR IMPACTS OF PROJECT INDEPENDENCE ON EARNINGS AND AFDC PAYMENTS, BY COUNTY 



Average To tal Earnin gs _ 



Aver age Total AFDC Payments 



Program 

County and Follow-up period Gr oup ($) 



Bay 

Years 1 -2 

Year 1 
Year 2 



(sample size = 459) 



5501 

2549 
2953 



Control 
Group ($). 



Difference 



5273 

2143 
3130 



229 

406 
-177 



Percentage 
Change 



4.3% 

18.9% 
-5.7% 



Program 
Group ($) 



Control 
Group ($) 



Difference 



4099 

2512 
1587 



3994 

2674 
1320 



106 

-162 
267 



Percentage 
Change 



2.6% 

-6.1% 
20.3% 



Broward (sample size = 2,240) 



Years 1 -2 


6912 


6443 


469 


7.3% 


3479 


3793 


-314 ** 


-8.3% 


Year 1 


3117 


2713 


404 * 


14.9% 


1818 


2018 


-200 *** 


-9.9% 


Year 2 


3795 


3731 


65 


1 .7% 


1661" 


1774 


-113 


-6.4% 


)ade 


(sample size = 6,021) 
















Years 1 -2 


5129 


4881 


249 


5.1% 


4733 


4918 


-185 ** 


-3.8% 


Year 1 


2184 


2090 


93 


4.5% 


2523 


2603 


-80 ** 


-3.1% 


Year 2 


2946 


2790 


155 


5.6% 


2209 


2315 


-105 ** 


-4.5% 



Duval 

Years 1 -2 

Year 1 
Year 2 



(sample size = 2,016) 



5066 

2228 
2838 



5636 

2545 
3092 



-570 

-316 * 
-254 



-10.1% 

-12.4% 
-8.2% 



4241 

2314 
1927 



4464 

2454 
2010 



-223 * 

-140 ** 
-83 



-5.0% 

-5.7% 
-4.1% 



HiKsborough (sample size = 2,415) 

Years 1-2 

Year 1 
Year 2 



5489 5817 -328 

2454 2524 -71 

3035 3292 -257 



-5.6% 3719 

-2.8% 2080 
-7.8% 1639 



4112 -393 *** -9.6% 

2288 -207 *** -9.1% 

1825 -186 ** -10.2% 
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TABLE 6.13 (continued) 



Aver age To t al Earn ings 



Average Total AFDC Pavnnent s 



Counjy and Follow-up period 
Lee 



Program 
Group ($) 



(sample size = 687) 



Years i -2 

Yean 
Year 2 



6335 

2975 
3360 



Control 
Group ($) 



5002 

2377 
2625 



Difference 



1333 

598 
735 



Percentage 
Change 



26.6% 

25.1% 
28.0% 



Program 
G roup ($) 



3130 

1715 
1415 



Control 
Group ($) 



Difference 



Percentage 
Change 



3477 

2041 
1436 



-347 

-326 
-21 



-10.0% 

-16.0% 
-1.5% 



Orange 

Years 1 -2 

Year 1 
Year 2 



(sample size = 1 .749) 



6764 

3084 
3681 



5993 

2512 
3481 



771 * 

571 
200 



12.9% 

22.7% 
5.7% 



3647 

2036 
1611 



4123 

2333 
1790 



—477 *** 

-297 
-179 



-11.6% 

-12.7% 
-10.0% 



LA 



Pinellas 

Years 1 -2 

Year 1 
Year 2 



(sample size = 1 ,737) 



6802 

2987 
3815 



6332 

2750 
3582 



470 

237 
233 



7.4% 

8.6% 
6.5% 



3450 

1932 
1517 



3653 

2007 
1646 



-203 

-75 
-129 



-5.6% 

-3.7% 
-7.8% 



Volusia 

Years 1 -2 

Year 1 
Year 2 



(sample size = 909) 



5255 

2343 
2912 



4813 

2175 
2638 



441 

168 
274 



9.2% 

7.7% 
10.4% 



3542 

2036 
1506 



3786 

2126 
1660 



-244 

-90 
-154 



-6.5% 

-4.2% 
-9.3% 



ERLC 



All counties 

Years 1 -2 

Year 1 
Year 2 

SOURCE- 
NOTES. 



(sample size = 18,233) 



5766 

2548 
3219 



5539 

2401 
3138 



227 * 

146 ** 

80 



4.1% 

6.1% 
2.6% 



4028 

2196 
1832 



4293 

2348 
1945 



-265 *** 

-152 *** 
-113 



MDRC calculations from Florida Unemployment Insurance (UI) earnings records and AFDC records. 
See Table 6.2. 

Variation in two-year Impacts on earnings and AFDC payments across counties was not statistically significant at the 10 percent level. 



2 '-^4 



-6.2% 

-6.5% 
-5.8% 



presented in this report about the true magnitude, source, or policy importance of county variation in 
impacts. 

VII. Comparisons with Other Welfare-to-Work Programs 

To provide a context for gauging the magnitude of Project Independence's two-year impacts, 
it is useful to compare them with those generated by other JOBS programs and pre- JOBS programs. 
One important random assignment evaluation of a JOBS program for which there are comparable data 
is the study of California's Greater Avenues for Independence (GAIN) Program, which was conducted 
by MDRC. During the 1980s, MDRC also conducted random assignment evaluations of seven pre- 
JOBS welfare-to-work programs, which provide another set of comparisons. One must be cautious 
in comparing impacts for Project Independence and these programs because there are several important 
differences in the research designs used, program models, and the environments in the various states. 
These are discussed briefly below. 

First, as discussed in Chapters 1 and 5, random assignment for the Project Independence 
evaluation took place at the point of AFDC application or redetermination; in GAIN and some of the 
other evaluations, random assignment took place at orientation. As also discussed previously, it is not 
clear whether the earlier point of random assignment used in the Project Independence evaluation 
served to increase the impact estimates (by capturing more of the total program effect) or to "water 
down" the impact estimates (by including in the research sample individuals who did not attend 
orientation and thus did not participate in activities). In any case, the Project Independence research 
sample includes individuals who did not attend orientation and so would not have become part of the 
samples in GAIN and other smdies. 

Second, there is evidence that perhaps 20 percent of the Project Independence control group 
may have been exposed to the program, a condition that was generally not present in the other 
evaluations. Such crossover would have decreased the difference in treatment between program and 
control group members and thereby decrease the estimate of program impacts. In addition, control 
group members in the Project Independence evaluation were given access to subsidized child care and 
tuition assistance under the same guidelines and priorities as program group members, which tended 
to decrease the treatment difference between program and control group members. This was generally 
not the case in the other evaluations. 

Third, the research sample for Project Independence differed in several important respects from 
those of the other evaluations. The Project Independence evaluation research sample included single- 
parent AFDC applicants and recipients whose youngest child was between the ages of three and five 
(as well as those whose youngest child was six years old or older), whereas the research samples for 



^It is worth noting that program administrators, in advance of the evaluation, picked Lee County as the 
one likely to have the largest impacts. That the large earnings impacts for Lee came out in accord with their 
expectations suggests that administrators can identify superior performance across a number of program sites. 
This would be an important finding, if it could be replicated, smce other studies have indicated that the 
measures of performance traditionally relied on by administrators are not reliable (e.g., see Friedlander, 
1988). 
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most of the other evaluations did not include this group. To increase comparability, this section 
focuses on Project Independence results for single-parent AFDC applicants and recipients whose 
youngest child was age six or older at the time of random assignment. It may also be noted that 
the Project Independence research sample included a much higher proportion of AFDC applicants and 
reapplicants than did most of the other research samples. Applicants and reapplicants made up 83 
percent of the Project Independence research sample. Only one of the other programs in this 
comparison exceeded this rate, and most were far below it.^^ GAIN had a much lower rate (only 
22 percent for the research sample as a whole), and zero percent for two of the research counties. 
This means that research sample members in the Project Independence evaluation may have been 
somewhat less "disadvantaged," on average, than was typical in the other evaluations. 

Fourth, the evaluations differed in their state AFDC grant levels. In particular, Florida ranks 
in the lower range of states across the country. In this respect, it differs especially from California, 
which ranks among the highest. For example, in January 1991, the basic AFDC grant in Florida for 
a family of three was $294, and 35 states had higher grant levels. In that same month, the basic 
AFDC grant for a family of three in California was $607; only Alaska had a higher grant level. This 
difference means that Project Independence and GAIN participants faced very different incentives and 
opportunities to supplement or replace welfare with earnings. It may also be noted that labor market 
conditions differed across evaluations. 

Fifth, the other evaluations were conducted in earlier years, and inflation has changed the value 
of the dollar. A given dollar amount of impact from the earlier programs would be worth more in 
today's dollars. For GAIN, inflating the evaluation impacts to current dollars would have only a 
minor effect. But dollar amounts from most of the other program evaluations would increase by about 
one-third above those shown below. It is not clear that such an inflation adjustment would yield 



^^The Arkansas program (which pre-dated JOBS) did include sample members whose youngest child was 
between the ages of three and five, and individuals in that group are included in the estimates presented below 
for that program. Three counties in the GAIN evaluation - Alameda, Riverside, and Tulare - also worked 
with this group, but most of those individuals were brought into the program after the start of the evaluation, 
and this subgroup is not included in the basic GAIN estimates presented later in this section. Estimates for 
this subgroup from these three GAIN counties and ft*om Project Independence are compared after the main 
discussion. 

^^San Diego I (the San Diego Employment Preparation Program/Experimental Work Experience Program, 
or EPP/EWEP) worked only with applicants. 

^^The percentage of AFDC applicants and reapplicants in the GAIN research sample was calculated as 
the average of the percentages in each of the six GAIN research counties, which were weighted equally (see 
Table 1 .2 in Riccio, Friedlander, and Freedman, 1994). Alameda and Los Angeles did not include any AFDC 
applicants or reapplicants in their research samples; those counties worked only with long-term AFDC 
recipients. 

^^Data on earnings and AFDC payments for the Project Independence evaluation came largely from the 
period 1992-1993. The year 1993 can therefore be taken as the "current year." An appropriate year upon 
which to base inflation adjustments for GAIN would be 1992. For most of the other evaluations cited later 
in this section, an appropriate base year would be 1985. Louisville group and individual job search were 
evaluated earlier, but the estimates shown for them below have been inflated to 1985 dollars. For San Diego 
SWIM, the appropriate base year would be 1988. Inflating GAIN results from 1992 to 1993 requires that 

(continued...) 
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the most appropriate comparison figures, however. AFDC grant levels and the minimum wage have 
increased at a slower rate than general inflation. Adjustment for inflation in the usual fashion would 
therefore not account for the levels of AFDC and the wage rates available to the welfare populatiorx. 

Finally, the programs differed in the relative emphasis they placed on immediate labor force 
attachment and human capital development. They also differed in the amount of resources available 
and in their targeting approaches. Most of the programs were relatively low-cost, consisting mainly 
of job club or other job search assistance, often including a work experience activity as well. Florida 
relied heavily on very low-cost independent job search, but Florida and three other programs also 
added some more costly education and training to the service mix. San Diego SWIM put those more 
expensive activities at the end of a sequence that began with job search and work experience. Balti- 
more Options allowed caseworkers to assign education or training to enroUees whom they deemed 
appropriate for it, and also allowed enrollees latitude to choose those activities. GAIN placed the most 
emphasis on basic education, making it a mandatory first activity for enrollees who scored low on a 
basic skills test or had no high school diploma or GED.^-^ 

The above cautions should be kept in mind when comparing results for Project Independence 
to those from other evaluations. Table 6.14 presents short-term impacts on earnings and AFDC 
payments for single-parent AFDC applicants and recipients for all the programs. Results for all the 
programs listed in the top portion of the table pertain to the subgroup without preschool-age children. 
Project Independence is shown first in the table (with a breakdown for the early and late cohorts), 
followed by GAIN and the other programs, which are shown in order of decreasing net program cost. 
GAIN was the most costly program, with net costs of $3,422 per sample member (about $3,500 in 
present dollars). Baltimore Options and San Diego SWIM were in the middle range, with net costs 
just under $1,(X)0, uninflated (about $1,200 to $i,300 in present dollars). The other programs were 
less expensive, in the range of $100 to $600 per sample member, uninflated (from about $150 to about 
$800 in present dollars). Estimates for the subgroup with preschool-age children are shown at the 
bottom of the table. 

To sumumarize: A wide range of impacts on AFDC have been measured in previous evaluations 
of welfare-to-work programs. The 6 percent reduction in two-year AFDC payments measured for 



^^(... continued) 

dollar amounts be multiplied by 1.03. Inflating San Diego SWIM results from 1988 would require 
multiplication by 1.20, and inflating the other program results from 1985 would require multiplication by 
1.33. 

Although gain's service emphasis varied substantially by county, on average, the program placed a 
greater emphasis on basic education than did Project Independence, San Diego SWIM* or Baltimore Options 
(see Riccio, Friedlander, and Freedman, 1994). 

Although not shown in ihe table, the crossover adjustment described in Chapter 5 may be applied to any 
Project Independence estimate by dividing 1 minus .198, which is approximately the same as multiplying by 
1.25. This crossover adjustment may correct for one of the differences between the Project Independence 
evaluation and the others, namely, that significant numbers of conuol group members in Florida were exposed 
to the program treatment. Crossover in the Florida evaluation v/ould tend to make Project Independence's 
impact estimates understate the program's true impacts, especially in comparison with other evaluations where 
crossover did not occur. 
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TABLE 6.14 



IMPACTS FOR SINGLE-PARENT AFDCs IN RANDOM ASSIGNMENT EVALUATIONS 
OF STATE WELFARE-TO-WORK PROGRAMS 



Average Annual Earnings ($) Average Annual AFDC Payments ($) 
Subgroup and Program Net Cost Year 1 Year 2 Year 3 Year 1 Year 2 Year 3 



Youngest child, age 6 or older 



Project Independence 
Early cohort (a) 
Late cohort 


n/a 
1128 
877 


284 *** 
278 * 
307 ** 


190 
211 
181 


n/a 

424 ** 
n/a 


-171 
-139 *** 
-192 *** 


-133 
-105 * 
-155 


n/a 
-101 * 
n/a 


California GAIN (b) 


3422 


266 *** 


512 


636 


-283 *** 


-347 *** 


-331 ♦** 


Baltimore Options 


953 


140 


401 *** 


511 *** 


2 


-34 


-31 


San Diego SWIM (c) 


919 


352 *** 


644 *** 


555 *** 


-419 


-560 *** 


-483 *** 


oan Uiego i (hPP/tWcP) 


636 


443 *** 


n/a 


n/a 


-226 *** 


n/a 


n/a 


Virginia ESP 


430 


69 


280 ** 


268 * 


-69 


-36 


-111 ** 


West Virginia 
CWEP 


260 


16 


n/a 


n/a 


0 


n/a 


n/a 


Louisville WIN Lab- 
Group Job Search (d) 


230 


464 ** 


n/a 


n/a 


-40 


n/a 


n/a 


Cook County 
WIN Demonstration 


157 


10 


n/a 


n/a 


-40 


n/a 


n/a 


Louisville WIN Lab- 
Individual Job Search (d) 


136 


289 ** 


456 ** 


435 ** 


-75 * 


-164 ** 


-184 


Arkansas WORK 
Program (e) 


118 


167 ** 


223 


337 ♦* 


-145 *** 


-190 *** 


-168 *** 


Youngest child, aqe 3~5 

Project Independence 
Early cohort (a) 
Late cohort 


n/a 
1831 
894 


54 
142 
14 


-34 
216 
-189 


n/a 
97 
n/a 


-133 *** 

-86 
-182 *** 


-77 * 
-117 * 
-59 


n/a 
-2 
n/a 


California GAIN 
Three counties (f) 


n/a 


512 (Z) 


732 (Z) 


711 (z) 


-108 (Z) 


-194 (Z) 


-350 (z) 



SOURCES: Gueronand Pauly, 1991; Riccio, Friedlander. and Freedman. 1994; Friedlander and Hamilton, 1993. 



NOTES: All estimates are uninflated unless otherwise noted. 

A two-tailed t-test was applied to the differences between program and control groups. Statistical 
significance levels are indicated as *** = 1 percent; ** = 5 percent; * = 10 percent. 

Where results are not available, "n/a" is used. 

(a) Year 3 earnings equal the sum for quarters 1 0 through 1 2 times 4/3. 

(b) Estimates are averages across six counties, with results for each county weighted equally. The counties are 
Alameda. Butte, Los Angeles, Riverside, San Diego, and Tulare. 

(c) All SWIM data in this table are taken from Friedlander and Hamilton, 1993. 

(d) Lousiville WIN Lab estimates are inflated to 1985 dollars. 

(e) Arkansas WORK Program estimates include a significant number of sample members with a youngest 
child 3 to 5 years of age. 

(f) Estimates are averages across three counties, with results for each county weighted equally. The counties 
are Alameda, Riverside, and Tulare. 

(z) Statistical tests are not available. 
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Project Independence falls roughly in the middle of the observed range of welfare savings. On the 
other hand, earnings impacts for Project Independence showed an atypical decline after the first year. 
Earnings impacts for the early cohort without preschool-age children do rank in the middle of the 
range established by other programs, but impacts for the other groups are lower. 

A detailed analysis requires an examination of the year-by-year results. Project Independence's 
first-year earnings impact of $284 for sample members without a child under age six was virtually 
identical to the first-year earnings impact for GAIN, It exceeded the first-year earnings impacts for 
Cook County, West Virginia, and Virginia. It was exceeded by the first-year impacts for both Louis- 
ville programs and both San Diego programs, and would be similar to the first-year earnings impacts 
for Arkansas and Baltimore if those were inflated to current dollars. Earnings impacts in year 1 are 
not tlie most important for purposes of comparison, however. In all six evaluations with follow-up 
beyond one year, impacts on earnings increased from year 1 to year 2, and impacts on earnings either 
grew from year 2 to year 3 or declined by less than 16 percent (as they did for San Diego SWIM). 
All of these programs featured a major job search component, and, in several, job search was by far 
the dominant activity. For example, in the Louisville Individual Job Search program, individual job 
search was the only activity, the net cost of the program was $136 per sample member (in 1985 
dollars), and tlie earnings impacts increased substantially from year 1 to year 2 and remained at almost 
their peak level in year 3, Similarly, in the Arkansas WORK program, group and individual job 
search were the program activities for almost all participants, the net cost of the program was $118 
(in 1985 dollars), and the earnings impacts increased substantially from year 1 to year 2 and again 
from year 2 to year 3. The evidence from past evaluations is that job search can be effective in 
increasing earnings, and the earnings effects can last at least three to four years before beginning to 
decline. 

In Project In dependence, earnings impacts for the subgroup without preschool-age children 
decreased about one-third between year 1 and year 2, For the early cohort in that subgroup, however, 
the substantial increase from year 2 to year 3 more than made up for that temporary decline. The 
impact for year 3 is the most important to compare across programs, since it is probably most repre- 
sentative of longer-term impacts. The third-year earnings impact for Project Independence is in the 
same range as third-year results for Virginia ESP, Louisville Individual Job Search, and Arkansas 
WORK, all of which were primarily job search programs. These estimates, when inflated, would form 
a range of approximately $350 to $600 for third-year earnings impacts. The third-year earnings impact 
for the Project Independence early cohort of the subgroup without preschool-age children is in this 
range. When the crossover adjustment is applied, the resulting third-year earnings impact estimate is 
still in this range. 

Two-year impacts on AFDC payments for Project Independence were $304 for the group 
without preschool-age children, a reduction of 7.5 percent of payments to the control group. For the 
early cohort of this subgroup, the two-year impact was $244, a reduction of 6,3 percent. These 
amounts are similar, and the third year for the early cohort does not show much change over time. 
Conclusions about the relative magnitude of Project Independence AFDC payments impacts are 
therefore less contingent on early-cohort results than is the conclusion about earnings impacts. As with 
earnings, the early-cohort estimates are taken as representative of Project Independence. The dollar 



^^See, e.g., Friedlander and Burtless, 1995. 
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amount of those savings was clearly exceeded by GAIN and San Diego SWIM. Dollar impacts in 
Arkansas, when inflated, would exceed those in Florida. Dollar impacts for San Diego I, when 
inflated and projected into the unavailable year 2, would also probably exceed Florida's. Inflated 
impacts for Louisville Individual Job Search would exceed Florida's. Florida clearly achieved larger 
AFDC savings than four of the other five programs. It also probably achieved larger AFDC impacts 
than Virginia, although longer-term follow-up would be helpful in determining the relative magnitudes 
of AFDC impacts for that comparison. Across all programs, then. Project Independence would be in 
the middle rank for dollar AFDC impacts. 

The direct comparison of impacts on AFDC payments discussed above is problematic owing 
to differences in AFDC benefit levels and differences in sample composition. The low AFDC benefit 
levels in Florida placed a limit on the dollar saving that could be achieved from any given case closing. 
In addition, the high proportion of AFDC applicants in the Project Independence sample meant that 
a large fraction of control group members would leave AFDC within a few years after random 
assignment. The average AFDC payment per control group member would be reduced by the large 
number of controls who left AFDC. A given percentage AFDC impact would therefore appear as a 
smaller average dollar impact than it would in a sample where a large fraction of controls continued 
to receive AFDC. It is therefore useful to examine impacts on AFDC payments as a percentage of 
payments to the control group. Comparing percentage AFDC effects is a way of adjusting for the 
differences in grant levels across states. When the question is how well a program has reduced AFDC 
expenditures for its own particular caseload, comparing percentage AFDC effects is more informative 
than comparing dollar effects.^^ 

The two-year AFDC impact in Project Independence for the early cohort of the subgroup 
without preschool-age children was 6.3 percent of control group average payments (not shown in Table 
6.14). This was similar to the two-year reduction achieved by GAIN (5.6 percent) and Louisville Indi- 
vidual Job Search (about 6 percent); larger than the saving in Virginia (about 3 percent); and slightly 
smaller than the saving in San Diego I (8 percent in year 1). The only substantially larger percentage 
reductions for two-year AFDC impacts were in San Diego SWIM (about 11 percent) and Arkansas 
(about 16 percent). In addition, the decline in dollar AFDC impact from year 1 to year 2 in Florida 
did not amount to a significant decline in percentage terms because the average payments to control 
group members fell, too. In fact, the percentage impact increased from year 2 to year 3 for the early 
cohort. Thus, converting AFDC impacts from dollars to percentages makes the impacts of Project 



-^^Applying the crossover adjustment to AFDC savings yields a two-year impact estimate of $304 for the 
early cohort of the subgroup without preschool-age children, which does not significandy change the ranking 
of dollar impacts in Project Independence relative to diose of the odier programs. The crossover-adjusted two- 
year AFDC impact for the full subgroup without preschool-age children is $379, which is also not large 
enough to change the relative ranking of the program. 

^^To see the utility of percentage effects, suppose that three programs have different fractions of enrollees 
who would be on AFDC in the iong term without intervention. Those fractions are 20 percent, 50 percent, 
and 80 percent. Further, suppose that each program is successful in inducing 20 percent of those potential 
long-term recipients to leave AFDC in the short term. The percentage point impacts would be 4, 10. and 
16, respectively (i.e.. .20 x 20, .20 x 50, and .20 x 80). The percentage AFDC effects would, however, 
all be 20 percent, indicating a 20 percent success rate in reducing long-term AFDC receipt for all three 
programs. 
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Independence appear larger when compared to the other programs, and percentage AFDC impacts are 
likely to continue to compare favorably over time.^^ 

Another way of comparing effects across programs is to examine the degree to which earnings 
increases replaced or exceeded reductions in AFDC payments. Given a similar percentage reduction 
in AFDC payments between two programs, the one with the higher ratio of dollars of earnings impact 
to dollars of AFDC payments impact could be judged the more effective. Over two years, earnings 
impacts for the subgroup without preschool-age children in Project Independence were 56 percent 
greater than the AFDC payments impacts. This ratio declined from year 1 to year 2. For the early 
cohort in this subgroup, however, the ratio was larger for the first two years and also improved with 
the addition of third-year impacts. In year 3, taken alone, earnings impacts for the early cohort in this 
subgroup were more than four times the size of the AFDC payments impacts. The cumulative ratio 
of earnings impacts to AFDC payments impacts may therefore show improvement over time. That 
kind of pattern of improvement over time was found in some other programs. For example, in GAIN, 
earnings impacts in year 1 were less than AFDC payments impacts, but by year 3 the former were 
almost double the latter. Moreover, earnings impacts in GAIN were still growing in year 3, and the 
cumulative excess over AFDC reductions is likely to increase in GAIN in later years. 

The preceding discussion pertains to impacts for the subgroup without preschool-age children. 
Findings for the Project Independence subgroup with preschool-age children are shown at the bottom 
of Table 6.4. This subgroup was generally not part of the mandatory target population for welfare-to- 
work programs prior to JOBS. Impact estimates for that subgroup, however, are available for three 
GAIN counties following the implementation of JOBS. The annual impact estimates for these three 
counties combined are shown in Table 6.14. Earnings impacts for this subgroup in GAIN grew from 
year 1 to year 2 and did not decline much from year 2 to year 3. Over the three years of GAIN 
follow-up, the total of the annual earnings impacts reached nearly $2,000. Impacts on AFDC 
payments were not especially large initially but were still growing in the third year of follow-up. 
These GAIN results indicate that it is possible to achieve large earnings impacts for this subgroup and 
perhaps large AFDC impacts as well. 

Achieving such results for the subgroup with preschool-age children may require more 
resources than were available to Project Independence. As shown in the table, earnings impacts in 
Project Independence were not large for this subgroup and declined substantially for both the early and 
late cohorts by the end of the follow-up period. Impacts on AFDC payments were found during the 
first two years of follow-up, but these effects had mostly faded out for both early and late cohorts by 
the end of the follow-up period. 

Finally, it is of interest to determine whether the difference in characteristics of sample 
members contributed to the relative ranking of Project Independence versus the other programs. For 
example, would the earnings impacts of Project Independence have been larger if the percentages of 



Applying the crossover adjustment raises the two-year percentage reduction to 7.8 percent for the early 
cohort of the subgroup without preschool-age children. The adjustment thus makes percentage AFDC impacts 
look slightly more favorable in comparison to other programs. The AFDC payments percentage reduction 
for the full subgroup without preschool-age children was 7.5 percent of the control group average. Crossover- 
adjusted, this figure was 9.4 percent. 
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the sample in each of the major subgroups had been more similar to those in the earlier program 
evaluations? For such an assessment, the distribution of the "level of disadvantage" subgroups in some 
GAIN counties is suitable. The "level of disadvantage" categories were defined in the same way in 
both studies. The distribution across these categories can be compared for GAIN and for the full 
sample (with and without preschool-age children) in Project Independence. In the GAIN samples, the 
level of disadvantage was generally greater than in the Project Independence sample. Impact estimates 
for the level of disadvantage categories in the Project Independence study can be averaged to produce 
a new overall impact estimate, giving each category the weight it had in a GAIN county. This 
increases the total earnings impacts of Project Independence by as much as 20 percent and reduces the 
rate of decline from year 1 to year 2 by about one-third. AFDC impacts are increased by a little less 
than 15 percent. Thus, sample characteristics probably have a role, but a relatively minor one, in 
explaining the impact results for Project Independence.^^ 



^Two GAIN counties were used in these calculations. Riverside aud San Diego. These were selected 
because they represented a middle level of disadvantagedness among the GAIN counties. The impact estimates 
and percent-of-sample weights for the level-of-disadvautage subgroups were obtained from Table 6.11 of the 
present report, which includes cases with and without preschool-age children. For the GAIN counties, 
percent-of-sample weights were obtained from Riccio, Friedlander, and Freedman (1994), p. 142. The impact 
estimates produced were as follows: 

Weighted Impact Estimates 



Eamings Impacts AFDC Payments Impacts 



Project Independence 
Riverside GAIN 
San Diego GAIN 



Years 






Years 






1-2 


Year 1 


Year 2 


1-2 


Year 1 


Year 2 


$280 


$183 


$97 


-$307 


-$172 


-SI 35 


$330 


$197 


$133 


-$343 


-$201 


-$143 


$336 


$200 


$136 


-$347 


-$204 


-$144 



These are the estimates supporting the discussion in the text. Note that the Project Independence estimates 
differ from those for the full sample presented in Table 5. 1 because they are weighted subgroup averages, and 
are not estimated directly from the full sample. Details may not sum to totals owing to rounding. 
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CHAPTER 7 



BENEFIT-COST ANALYSIS 



Drawing on the analysis of net costs presented in Chapter 4 and the analysis of Project 
Independence's impacts presented in Chapters 5 and 6, this chapter provides an overall accounting of 
the financial gains and losses produced by the program. It presents these results from the perspective 
of the AFDC applicants and recipients who were required to participate in Project Independence 
(referred to in this chapter as the "welfare sample"), the government budget, taxpayers, and society 
as a whole. The analysis of benefits, like the study of impacts, includes program-control group 
differences in the value of earnings, AFDC, and Food Stamps. However, it goes beyond these basic 
impact measures to consider Project Independence's effects on fringe benefits from employment, taxes. 
Unemployment Insurance (UI) benefits, Medicaid, and the cost of administering transfer programs. 
Cost estimates include the net costs of providing Project Independence and non-Project Independence 
employment-related services to program group members. As in Chapter 4, net costs were estimated 
by subtracting the average value of resoi- ces used by control group members from the corresponding 
average for program group members. 

The chapter begins by describing the scope of the analysis and the framework used. The next 
sections examine Project Independence's effect on earnings, fringe benefits, taxes, and transfer 
payments for the full research sample. These effects are then added together to produce a single 
measure of Project Independence's benefit-cost results from each of the four perspectives noted above. 
The chapter concludes with a brief discussion of the results for the cohort and age-of-youngest child 
subgroups. 



L Analytical Approach ^ 

The benefit-cost analysis places dollar values on the program's effects and its use of resources. 
It includes both measured effects on earnings, AFDC payments. Food Stamps, and Unemployment 
Insurance benefits and imputed effects on fringe benefits, state and federal taxes. Medicaid payments, 
and the costs of administering transfer programs. The analysis uses earnings and transfer payment 
records in combination with transfer payment eligibility rules, tax regulations, published data from 
state and federal agencies, and other sources to calculate imputed values. 

A. Accounting Methods 

The primary benefit-cost estimates presented in this chapter cover a five-year time horizon 
starting with the first quarter after the quarter of random assignment (quarter 2). This time frame is 



^The benefit-cost analytical approach used in the Project Independence evaluation is virtually the same as 
that used in previous MDRC evaluations, and many of the techniques employed were originally developed 
for the evaluations of state programs in MDRC's Demonstration of State Work/Welfare Initiatives. (See Long 
and Knox, 1985, for additional information.) This report's description of that ^proach is adapted from 
Riccio, Friedlander, and Freedman, 1994 (the final report from MDRC's evaluation of California's GAIN 
program). Minor distinctions have been introduced because of the data that were available for the present 
evaluation and the unique features of Project Independence and its contexts. 
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similar to the one used in most previous MDRC evaluations of welfare-to-work programs that 
emphasized short-term, job-search-oriented activities and where effects were expected to occur quickly 
and then decrease over time. 

The five-year time horizon includes an observation period and a projection period. The 
observation period for each sample member encompassed the portion of follow-up when benefits were 
estimated (or imputed) directly from "obser\'ed" - i.e., recorded - earnings and transfer payments 
data. It extended from quarter 2 through the last month of available data (September 1993) and 
covered two full years for the full sample and up to three years for those randornly assigned during 
the first month of the study - July 1990. (It should be noted that 69 percent of the sample had at least 
two and a half years of follow-up in the observation period.) Gains and losses observed at the end of 
this period were then projected to the end of year 5, using several assumptions about the size of future 
effects. For the Project Independence evaluation benefit-cost analysis, the projection period ranges 
from two to three years. In general, benefit-cost estimates based primarily on observed data are more 
reliable because they are less affected by assumptions about how costs and benefits are likely to change 
in the projection period. Projecting estimates for this study is particularly complicated because, as 
discussed earlier, the implementation of Project Independence and the state and local environments in 
which the program was operated underwent several very important changes during the evaluation 
period. The analysis attempts to account for the effect of some of these changes by making different 
projection assumptions for different cohort subgroups in the research sample. However, it is important 
to recognize that the combination of changes exacerbates the uncertainty of the projected estimates both 
on the cost side (as discussed in Chapter 4) and on the benefits side. At the same time, this 
uncertainty is not so extreme as to challenge the general pattern of benefit-cost results presented in this 
chapter. 

The main findings of the analysis are expressed in terms of net present values per program 
group member. "Net*" means that the amounts represent differences between program and control group 
members, just as impacts do. "Present value" is an accounting method for estimating the worth today 
of monetary effects that occur in the future. 

In a welfare -to-work program such as Project Independence, most costs are incurred early on, 
particularly in the first two years, when service use is heaviest, while many benefits (e.g., earnings 
gains and welfare savings) continue to be realized in later years. However, simply comparing the 
nominal dollar value of program costs and benefits would be problematic. The value of a dollar is 
greater in the present than in the future: A dollar available today (either to program group members 
or to the government) can be invested and produce income over time, making it worth more than a 
dollar available in the future. Thus, to make a fair comparison between costs and benefits, it is 
essential to focus on their value at a common point in time - i.e., in the present. 

The benefit-cost analysis addresses this issue by discounting, i.e., by adjusting the value of 
benefits accruing after the program "investment period" to reflect their lower value in terms of the time 
when program costs were incurred. In effect, an estimated amount of interest income foregone must 
be subtracted from the nominal value of the benefits occurring after the investment period.^ 



^Put differently, a benefit occurring at time 2 has the same value as a smaller benefit occurring at time 
1 plus interest, thus, subtracting the interest income from the time 2 benefit yields its value at time I . 
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This analysis uses the end of the first year following random assignment (quarter 5) as the 
comparison point for the investment period, since it is about mid-way in the period in which most of 
the program and control group members' participation in employment-related activities occurred (and, 
accordingly, mid- way in the period when many costs were incurred, as noted in Chapter 4), Thus, 
gains that were accrued later were discounted to reflect their value at the end of quarter 5. In 
calculating these discounted values, it was assumed that a dollar invested at the end of quarter 5 would 
earn a real rate of return of 5 percent aimually.*^ Furthermore, all benefits and costs are expressed 
in 1993 dollars, eliminating the effects of inflation, 

B» Analytical Perspectives 

Once estimated, particular net benefits and net costs will constitute gains or losses, or be 
irrelevant, depending on which of the analytical perspectives - the welfare sample, the government 
budget, taxpayers, or society - is considered/ The welfare sample perspective identifies net gains 
or losses for members of the program group, indicating how they fared as a result of the program.^ 
As illustrated by the in-text box that follows shortly, earnings impacts represent gains for the welfare 
sample, while reductions in AFDC (and otb.er transfers) represent losses,^ Higher taxes paid by 
program group members compared to contro] .^xoup members also constitute losses to the welfare 
sample. In essence, a program produces a net gain from the standpoint of the welfare sample if 
program group members* earnings gains exceed the value of reductions in transfer payments and 
higher taxes7 The net costs of providing employment-related services to program group members 
have no direct effect on their income and are not considered as net gains or losses from the perspective 



^For example, if a welfare-to-work program increased revenues to the government budget by an average 
of $1,221 per program group member in the last quarter of year 5, its net present value would be $1,000 from 
the standpoint of the investment period. That is because $1 ,000 invested at the end of quarter 5 at a 5 percent 
annual rate of interest (compounded continuously) equals $1,221 at the end of year 5. 

"^See Friedlander and Gueron, 1992; Friedlander and Hamilton, 1993; and Riccio, Friedlander, and 
Freedman, 1994, for other examples of these analyses, 

^The analysis does not take into consideration any effect Project Independence may have had on any 
"under the table" earnings of program group members. It also does not distinguish (or make any value 
judgments about) reductions in AFDC payments resulting from sanctions for noncompliance versus reductions 
owing to increased earnings or to case closures without earnings or sanctions; it counts all reductions in 
AFDC payments as losses to the welfare sample and savings for government budgets. 

^In this analysis, net increases in support service payments to program group members are not considered 
to be gains from the perspective of the welfare sample. These payments for child care, transportation, and 
ancillary expenses simply offset additional costs to program group members resulting from Project 
Independence's participation requirements. However, the analysis does include these payments as costs 
incurred from the government budget and taxpayer perspectives. 

^It follows that one program may produce higher earnings gains than another, but that the second may still 
show more positive benefit-cost results from the standpoint of the welfare sample. This result will occur if 
the second program produces smaller welfare reductions and increases in tax payments than the first. See, 
e.g. , the comparison of earnings gains and AFDC reductions recorded by the San Diego SWIM and Baltimore 
Options programs in Friedlander and Gueron, 1992, pp. 24-33. Put differently, a program produces a net 
gain from the standpoint of the welfare sample if program group members' total estimated income (the sum 
of earnings and transfer payments, plus the Earned Income Tax Credit, minus taxes) exceeds that of control 
group members. 
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of the welfare sample. Similarly, any budgetary savings in administering transfer programs have no 
direct effect on the welfare sample. 

The government budget perspective identifies net gains and losses incurred by federal, state, 
and local governments combined. Net gains to the government budget occur through savings in 
transfer payments and their related administrative costs and through higher taxes paid by program 
group members compared to control group members. The government budget comes out ahead to the 
extent that tax increases (resulting from earnings gains) and savings in transfer payments and 
administrative costs exceed the net cost of providing employment-related services to program group 
members. In and of themselves, program group members' earnings gains do not affect the calculations 
of net gains or losses from the standpoint of the government budget. 

The taxpayer perspective identifies benefits and costs from the standpoint of everyone in society 
other than individuals in the AFDC sample.^ Estimates of net gains and losses from the taxpayer 
perspective closely resemble those from the government budget perspective. The two perspectives 
differ only in the treatment of Social Security and Medicare taxes (and, in evaluations that include 
unpaid work experience, net gains from output that program group members produce in such 
assignments). Specifically, the government budget gains from both the welfare sample's and their 
employers' contributions to the two payroll taxes (i.e.. Social Security and Medicare), while taxpayers 
(who include employers) gain only from employee contributions. (Also, only taxpayers would gain 
from program group members' output in unpaid work experience assignments. However, this effect 
was not estimated for this evaluation because, as noted in Chapter 3, this activity was not used by the 
evaluation's research sample.) 

This analysis assumes that no displacement occurred as a result of employment gains by 
program group members. Rather, because displacement could not be measured, it assumes that 
employment gains for program group members represented an increase in the total level of employment 
and value of output. Alternatively, one could assume that at least a portion of program group 
members' employment gains (and earnings increases) occurred because program group members took 
jobs that would have gone to other members of society, leaving those individuals unemployed and 
possibly causing some of them to use government transfer programs. To the extent that this occurred, 
it would reduce the program's overall return for government budgets and taxpayers. 

As suggested by the above discussion, the results from the perspectives of the welfare sample, 
government budget, and taxpayers may be complementary, or they may conflict. One group's gains 
may appear as another group's losses. The accompanying box helps to illustrate this point. Here, a 
reduction in AFDC use would translate into a loss for the welfare sample and a corresponding gain 
for the government budget and taxpayers. However, an increase in earnings would reflect a gain to 
the welfare sample, but not to the government budget - although any taxes paid on those earnings 
would be a gain for the government budget. The net cost of employment-related services (e.g., 
education and training), in contrast, would be a loss to the government budget while leaving the 
welfare sample unaffected. 



^The term "taxpayer" is used for convenience and for the sake of consistency between this analysis and 
previous benefit-cost analyses. It should be noted that all members of the program group pay sales taxes and 
that many pay income and Social Security taxes as well. 
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Effect 


Welfare 
Sample 
Perspective 


Government 
Budget and 
Taxpayer 
Perspectives 


Societal 
Perspective 


Increase in earnings 


+ 


0 


+ 


Increase in income 
tax payments 






0 


Reduction in AFDC 
navments 






0 


Net cost of 
employment- 
related services 


0 







A welfare-to-work program might also produce net gains from the welfare sample, government 
budget, and taxpayer perspectives simultaneously, or net losses from all three perspectives.^ When 
the results are mixed (i.e., positive from some perspectives but not from others), an overall assessment 
of the program's merits depends upon one's willingness to value one perspective more highly than the 
others. Some will consider a program that increases the income of welfare recipients to be successful, 
even if taxpayers and the government budget realize some net loss. Others may judge a program as 
successful only if it produces budgetary savings. 

The final perspective, the perspective of society as a whole, combines the perspectives of the 
welfare sample and taxpayers (and the government budget). For a given component in the analysis, 
a net gain to society occurs only when a gain to one group is not at the expense of another group. For 
example, earnings gains for program group members represent a gain to the welfare sample without 
affecting taxpayers; thus, they are counted as a net gain to society. Net losses to society occur when 
what is a loss from one perspective is not a benefit from another. For example, the net costs of 
services represent a loss to the taxpayers and government budget but do not affect welfare recipients. 
Program effects that constitute a net gain from one perspective but a net loss from another (such as 
AFDC savings) have no financial consequences from the societal perspective - these effects, which 
represent a transfer from one group in society to another, simply cancel each other out. Thus, from 
the standpoint of society, a welfare-to-work program such as Project Independence would be judged 
successful in benefit-cost terms if it produced earnings gains for the welfare sample (these do not affect 
the government budget or taxpayers) and savings in transfer payment administrative costs (these do 



^There will be net gains from all three perspectives when earnings gains exceed the reductions in transfer 
payments and increases in taxes (a net gain from the perspective of the welfare sample), and the reductions 
in transfer payments and administrative costs, combined with increased tax reven-es, exceed the net cost of 
providing employment-related services (a net gain from the perspectives of taxpayers and the government 
budget). It is also possible for a program to produce net losses from all three perspectives (e.g. , when welfare 
savings plus tax increases exceed earnings gains, but net costs are higher still). 
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not affect welfare recipients) that together exceed the net cost of services. It should be noted that, 
when adopting the societal perspective, one assumes that the "value," or importance, of a dollar lost 
by one group is equivalent to that of a dollar gained by another group, which may or may not be a 
valid assumption. 

C. Limitations of the Analysis 

Some limits on the comprehensiveness of the benefit-cost analysis should also be recognized. 
In addition to the fact (already noted) that the estimates below do not take into account possible 
displacement of other workers by any increased employment of program group members, they do not 
include the value of the clear, but difficult-to-monetize, benefits associated with society's preference 
for work over welfare. Also, they do not place a dollar value on the foregone personal and family 
activities that are replaced by working or the intrinsic benefits of education that are not reflected in 
earnings. As is typical in benefit-cost analyses, certain effects cannot be quantified, and long-run 
effects cannot be gauged precisely. 



11. Program Effects (Benefits) for the Full Sample 
A. Earnings and Fringe Benefits 

Chapter 5 showed that Project Independence produced increases in work and earnings by 
program group members (compared to the control group) during the two-year follow-up period used 
for the impact analysis. Table 7. 1 presents the net present value of these earnings gains over the entire 
observation period (which, as previously noted, ranged from two to three years). As the table 
shows, the earnings gain for that period was $365 per program group member (in 1993 dollars). 

Fringe benefits - in the form of employer-paid health and life insurance, pension 
contributions, and worker's compensation associated with these earnings ~ were part of sample 
members' total compensation from working, and are included in the analysis. Using published data, 
these were estimated at the rate of 14.8 percent of wages. Thus, the average increase in earnings 



^"Tables 7.1 through 7.4, like the cost and impact tables, show program-control group differences as 
positive when the mean (i.e., average) value for program group members exceeds the mean value for control 
group members and as negative when the control group mean is higher. Table 7.5, which incorporates the 
four analytical perspectives, uses a different format for displaying benefit-cost results. In this table, an effect 
has a positive value if it represents a net gain from the perspective in question and a negative value if it 
represents a net loss. Therefore, the same effect will appear positive in some columns and negative in others. 

^ ^Earnings gains and reductions in AFDC and Food Stamps are discounted and expressed in 1993 dollars. 
In addition, the observation period used in the benefit-cost analysis differs somewhat from the two-year follow- 
up period used in the impact analysis. Therefore, program-control group differences in these measures differ 
from the value of impacts displayed in earlier chapters, even for the two-year follow-up, because of these 
transformations. 

^^U.S. Bureau of the Census, 1993, p. 430. The estimated value of fringe benefits was calculated as a 
ratio of the combined costs of employer-provided life and health insurance, retirement and pension accounts, 
and worker*s compensation to the combined costs of regular wages» paid leave (e.g., vacation and sick days) 

(continued...) 
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TABLE 7,1 



ESTIMATED PROGRAM -CONTROL GROUP DIFFERENCES IN EARNINGS. FRINGE BENEFITS, 
AND PERSONAL TAXES FOR THE OBSERVATION PERIOD, 
PER PROJECT INDEPENDENCE PROGRAM GROUP MEMBER 
(IN 1993 DOLLARS) 





Program 


Control 


- 


Component of Analysis 


Group 


Group 


Difference 


Earnings 


7729 


7365 


365 


Fringe benefits (a) 


1144 


1090 


54 


Total earnings and fringe benefits 


8873 


8454 


419 


Personal taxes 








Social Security payroll tax (b) 


591 


563 


28 


Federal income tax 


"485 


-454 


-31 


State income tax (c) 


n/a 


n/a 


n/a 


State sales and excise tax 


259 


257 


2 


Total taxes 


365 


367 


-2 


Sample size (total = 18.233) 


13.509 


4.724 





SOURCES: MDRC calculations from the State of Florida Unemployment Insurance (Ul) earnings and benefits records, 
and from published data on tax rates and employee fringe benefits. The end of the observation period was September 
1993 for all outcome measures. 



NOTES: Estimates reflect discounting and adjustment for inflation. 

Differences are regression-adjusted using ordinary least squares, controlling for pre-random assignment 
characteristics of sample members. 

Tests of statistical significance were not performed. 

Rounding may cause slight discrepancies in calculating sums and differences. 

(a) These include employer-pnid health and life insurance, pension contributions, and workers' compensation. 

(b) Employee portion only. 

(c) Florida does not have a state income tax, as indicated by "n/a" (not applicable). 
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of $365 per program group member plus an additional $54 in fringe benefits yielded an average 
increase in total work-related compensation of $419 per program group member during the observation 
period. 

B. Tax Payments 

Project Independence produced an increase in earnings, with effects on federal income taxes, 
payroll taxes, and state sales and excise taxes. Tax rates and rules for 1992, including the Earned 
Income Tax Credit (EITC),^"* were applied to an appropriate income base to impute taxes from 
earnings and other income. As shown in Table 7.1, total taxes per program group member actually 
decreased very slightly ($2) during the observation period. Interestingly, Project Independence 
decreased the level of federal income taxes paid by program group members despite their earnings 
gains because many program and control group members owed no federal income taxes once the value 
of standard deductions and exemptions was subtracted from their taxable income. Program group 
members also received larger EITC tax subsidies than control group members, which further offset 
potential tax increases. Increases in Social Security and Medicare payroll taxes were more than offset 
by the decrease in federal income taxes. 



^'^(... continued) 

and other benefits, which include severance pay, and supplemental (employer-provided) unemployment 
benefits. (Payments for leave time are captured directly by the earnings data and thus are not counted as a 
fringe benefit in this analysis.) The nimierator in this ratio represented 12.0 percent of employer costs in 
1992, while the denominator represented 81.1 percent. Dividing the second term into the first yields the 
fringe benefit rate of 14.8 percent used in this analysis. Legally mandated employer contributions for Social 
Security and Medicare were treated as taxes and were included later in the analysis. 
^^Horida does not have a state income tax. 

^"^The Earned Income Tax Credit (EITC) is a credit against federal income taxes for taxpayers with annual 
earnings below a threshold level. As with other tax credits, each dollar of EITC reduces by a dollar the taxes 
owed. Eligible persons can receive EITC as a payment from the government if they owe no federal income 
taxes. Schedule Z EITC rates for 1992 were used in this analysis. For 1992, only taxpayers who had 
dependent children and whose earnings ranged from $1 to $22,373 were eligible for the EITC. Taxpayers 
who had two or more children and who earned between $7,520 and $11 ,840 received the maximum value of 
the EITC, $1,384; those with only one child received up to $1,324. Taxpayers earning between $1 1,840 and 
$22,373 received a progressively lower value of the EITC. Not all eligible taxpayers receive the EITC. 
Followmg the approach MDRC adopted in its evaluation of California's GAIN program (Riccio, Friedlander, 
and Freedman, 1994), the EITC "take-up" rate was set at 70 percent. The GAIN analysis based this rate on 
findings from Scholz, 1994, and subsequent conversations with the author. The rate was applied to all sample 
members. Tnat is, each sample member's earnings were used to calculate the value of the EITC that she 
would have received; that v:»:ount was then multiplied by .7. 

^^Total earnings plus Unemployment Insurance compensation were used in computing federal income taxes 
for every sample member. The combined income from earnings, AFDC payments, and Unemployment 
Insurance compensation was used in calculating sales and excise taxes. The estimation of federal taxes are 
based on 1992 tax rates, exemption amounts, and Earned Income Tax Credit rules, since that year was about 
midway in the 1990-93 period of data collection for the earnings and AFDC data analyzed in this report. 

^^Employers pay an "employer's share" of these payroll taxes, which matches the rate paid by their 
employees. Therefore, the same increase in these payments by employers ($28 per program group member) 

(continued...) 
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C. Transfcr Payments 



The impact analysis presented in Chapters 5 and 6 found that Project Independence produced 
modest savings in AFDC payments and smaller savings in Food Stamps during the common two-year 
follow-up period. The benefit-cost analysis estimates the effects of Project Independence on these two 
transfer payments and also considers its effects on Unemployment Insurance (UI) benefits. Medicaid 
payments, and the costs of administering all four transfer programs. This section discusses the results 
of this analysis during the two- to three-year observation period. As before, program-control group 
differences are expressed in 1993 dollars and discounted to the end of quarter 5. 

As shown in Table 7.2, Project Independence realized an average of $300 in AFDC savings 
per program group member and $145 in Food Stamp savings per program group member during the 
observation period. Unemployment Insurance payments actually increased slightly ($27). 

Program-control group differences in average Medicaid payments were imputed on the basis 
of observed differences in AFDC receipt and earnings, rules governing Medicaid eligibility, and 
published data on average Medicaid payments made to all eligible individuals. An individual on 
AFDC is automatically entitled to receive Medicaid and, under certain circumstances, is eligible to 
receive transitional Medicaid for 12 months after leaving the AFDC rolls for employment.^ The 
analysis estimates program-control group differences for AFDC-related and transitional Medicaid and 
then combines these effects into a single estimate of Medicaid savings. To impute the value of 
Medicaid payments for AFDC recipients, it is necessary, first, to estimate the average value of 
Medicaid dollars paid on behalf of the sample member's AFDC case during a typical month of 
Medicaid eligibility. This average is calculated by multiplying the average Medicaid monthly payment 
for a single adult or child on AFDC by the number of adults and children on the sample member's 
case (as recorded at random assignment). (In 1990-91, in Florida, the average value of Medicaid 
payments per AFDC eligible person per month was $117.)^^ This average is then multiplied by the 
total number of months of AFDC receipt for each sample member, resulting in an estimate of total 
Medicaid expenditures for the observation period. Finally, the program-control group difference in 
total payments is calculated. A similar strategy was used to estimate Project Independence's effects 
on transitional Medicaid payments (based on the number of months of imputed eligibility for those 
payments). 



^°(... continued) 

was estimated for the analysis (but was not included in Table 7.1). Employer contributions do figure in the 
benefit-cost results from the perspective of the government budget. See Table 7.5. 

^^Estimates of the value of transitional Medicaid for the entire follow-up period are based on eligibility 
rules in effect since April 1990. Former AFDC recipients and their families can receive up to 12 months of 
transitional Medicaid if they lose AFDC eligibility because of increased earnings, increased hours of 
employment, or loss of earnings disregards. A basic requirement for transitional Medicaid eligibility is that 
a person receive earnings high enough to terminate her AFDC eligibility. (A person would not be eligible 
for transitional Medicaid if her gross monthly earnings minus necessary child care costs exceeded 185 percent 
of the federal poverty level. However » this requirement was not considered in the analysis because relatively 
few sample members earned that much.) Because only quarterly earnings data were available for the analysis, 
it was assumed that a sample member had met the earnings requirement in all three months of a calendar 
quarter if her quarterly earnings were at least three times the minimum level that, in a single month» would 
have closed her AFDC case. 

^^This information was obtained from Florida's Medicaid Services Budget Forecasting System Reports: 
Final Report, FY 1990-91. 
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TABLE 7.2 



ESTIMATED PROGRAM -CONTROL GROUP DIFFERENCES IN TRANSFER PAYMENTS 
AND ADMINISTRATIVE COSTS FOR THE OBSERVATION PERIOD. 
PER PROJECT INDEPENDENCE PROGRAM GROUP MEMBER 
(IN 1993 DOLLARS) 



Type of Payment or Cost 



Program 
Group 



Control 
Group 



Difference 



Transfer payments 
AFDC 

U! compensation 
Food Stamps 
Medicaid (while on AFDC) 
Transitional Medicaid 



4902 
321 
4897 
6158 
388 



5202 
295 
5042 
6489 
362 



-300 
27 
-145 
-331 
26 



Total 



16666 



17390 



-724 



Administrative costs 
AFDC 

Ul compensation 
Food Stamps 
Medicaid (while on AFDC) 
Transitional Medicaid 



819 
26 
741 
254 
16 



869 
24 
763 
267 
15 



-50 
2 

-22 
-14 
1 



Total 



1856 



1938 



-83 



Sample size (total = 18,233) 



13,509 



4,724 



SOURCES: MDRC calculations from the State of Florida Unemployment Insurance (Ul) earnings and benefits records. 
The end of the observation period was September 1993 for all outcome measures. 

NOTES: Estimates reflect discounting and adjustment for inflation. 

Differences are regression -adjusted using ordinary least squares, controlling for pre-random assignment 
characteristics of sample members. 

Tests of statistical significance were not perfomied. 

Rounding may cause slight discrepancies in calculating sums and differences. 
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As indicated in Table 7.2, Project Independence achieved savings in Medicaid (v/hile sample 
members were on AFDC) of $331. It is interesting to note that total Medicaid payments for program 
and control group members exceeded the value of their AFDC benefits. This is because, compared 
to other states, Florida has relatively low AFDC benefit rates and relatively high Medicaid payment 
rates. Not surprisingly, program group members received slightly hi^er levels of transitional 
Medicaid, on average ($26), reflecting their gains in earnings as well as decreases in their AFDC 
receipt. 

Combining average savings in AFDC, Food Stamps, and Medicaid with the small increase in 
average Unemployment Insurance benefits yielded an average savings in transfer payments of $724 
during the observation period. These savings, in turn, decreased the costs of administering transfer 
payments by an average of $83 per program group member. Project Independence's effects on 
transfer program administrative expenditures were estimated based on differences in use of the 
transfers and on information about state and federal program costs.^^ 

Future Effects 

Thus far, only program effects that occurred during the two- to three-year observation period 
have been considered. However, as discussed above, these effects are likely to last beyond this period, 
an expectation that should be taken into account in the analysis. Effects are consequently projected 
for each sample member beyond what was actually observed, so that the measured and projected 
effects together cover five years from the first quarter of the follow-up period (quarter 2). As 
discussed above, all sample members have at least two years of projected data, and the last individuals 
to enter the research sample have three years. 

Projecting program effects entails calculating a base period estimate and then making an 
assumption about how it will change in the future. This evaluation used data from each sample 
member's last four quarters of available follow-up to estimate Project Independence's base period 
effects. Assumptions then had to be made about the future effect of Project Independence for these 
individuals through the end of the five-year period. 

The first assumption for this analysis was that Project Independence's impacts on AFDC 
payments and Food Stamp payments would decrease at a rate of 25 percent per year during the 
projection period. This was based on the patterns observed for the full sample, which indicated that 
AFDC and Food Stamp impacts declined somewhat in the second year of follow-up and appeared 



^^For example, in 1991, the average monthly AFDC payment in Florida for a family of three was $294 
compared to the national average of $400. 

^^Combined federal, state, and local administrative costs for each of the four transfer payments were 
estimated as a percentage of the value of the payments, i.e., by dividing total administrative costs by total 
payments. The estimated percentages were 16.7 (AFDC), 15.1 (Food Stamps), 8.2 (Unemployment Insurance 
benefits), and 4.1 (Medicaid). Data for calculating these measures were obtained from tables in the U.S. 
Department of Health and Human Services, "Aid to Families widi Dependent Children (AFDC) Information 
Memorandum/ March 14, 1994, No. ACF-IM-94-1; U.S. Congress, House Committee on Ways and Means, 
Overview of Entitlement Programs, 1993; Florida Health Care Financing Administration, "Summary of 
Quarterly Medicaid Statement of Expenditures for the Medical Assistance Program (Form HCFA64)," 1994; 
and Florida Agency for Health Cai'e Administration, Medicaid Statistics, February 1992. 
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likely to keep doing so after that point. The analysis also assumes a 25 percent decay rate for 
Medicaid. 

Assumptions about the likely pattern of decay in earnings impacts were complicated by the 
divergent patterns observed for the cohort subgroups. Specifically, earnings impacts for the early 
cohort remained stable or increased slightly after the common two-year follow-up. However, the 
earnings impacts for the late cohort had already declined to less than zero at the end of two years (i.e., 
in that cohort, those in the program group eventually had slightly lower earnings than those in the 
control group). Thus, the projection of earnings impacts assumed a split decay rate: For the early 
cohort, the analysis assumes no decay in earnings impacts during the projection period; for the late 
cohort, the analysis assumes a 100 percent decay rate (i.e., no impacts on earnings and earnings- 
related outcomes) during the projection period. 

The resulting estimates are presented in Table 7.3. The values of all program effects have 
been discounted at a 5 percent real annual rate and adjusted for inflation to reflect 1993 dollars. Note 
that the $236 in projected earnings effects represent 39 percent of the total $600 five-year earnings 
increase. The projected amounts for AFDC, Food Stamp, and Medicaid impacts represent 
approximately 30 percent of the five-year totals. This means that projected earnings gains for program 
group members will not keep pace with their projected losses of AFDC, Food Stamp, and Medicaid 
benefits. However, as discussed below, the decrease in transfer payments for individuals represents 
savings from the government budget perspective. The projections in Table 7.3 indicate that the net 
gain to the government budget will grow in the future. 

The projected effects add some uncertainty to the overall five-year estimates, particularly for 
the earnings impact projections. For example. Table 7.4 shows that assuming a 45 percent decay rate 
for the full sample would yield a five-year earnings impact of $438 (27 percent less than the $600 
estimate using the split decay rate). It is also interesting to note that using a 0 percent decay rate for 
the full sample would yield a lower five-year earnings impact than the split decay rate. This is because 
the base period earnings impacts for the late cohort were slightly negative and, under the assumption 
of no decay, would have continued to be negative throughout the projection period. This makes the 
five-year earnings impacts lower than they would be assuming that there were no earnings impacts (as 
opposed to negative impacts) for the late cohort. 

For AFDC and Food Stamps, more extreme assumptions about impact decay rates (for 
example, 0 or 45 percent per year, rather than 25 percent, starting with the projection base amounts 
measured at the end of the observation period) would not change the five-year estimated effects by 
very much. In addition, using the different impact decay rate assumptions for earnings, AFDC, and 
Food Stamps would not change the general pattern of benefit-cost findings, which is discussed below. 



Some earlier studies of employment programs for welfare recipients have documented such a pattern 
and, in some cases, have indicated that program effects can actually increase over time. For examples, see 
the evaluation of the National Supported Work Demonstration (Masters and Maynard, 1981); the evaluation 
of a WIN job search program in Lx)uisv^ille, Kentucky (Wolfhagen» 1983); the evaluation of longer-term 
impacts of Options* a welfare employment program in Baltimore, Maryland (Friedlander, 1987); and the 
evaluation of longer-term impacts of the Arkansas WORK Program (Friedlander and Goldman, 1988). 
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TABLE 7.3 



ESTIMATED EFFECTS OF PROJECT INDEPENDENCE ON BENEFIT VARIABLES DURING THE OBSERVATION PERIOD. 
PROJECTION PERIOD, AND WITHIN FIVE YEARS AFTER RANDOM ASSIGNMENT, PER PROGRAM GROUP MEMBER 

(IN 1993 DOLLARS) 



Projection Period 

5 -Year Total 

Beneflt Variable Observ ation P eriod Proj ection Base (a) Projected Amount (Observed -f-Proiected) 



Earnings 365 

Fringe benefits 54 

Payroll taxes 

Employee portion 28 

Employer portion 28 

Income and sales taxes (b) -29 

AFDC payments -300 

Food Stamps -145 

Ul compensation 27 

Total Medicaid -305 

Transfer program administration -83 



21 236 600 

3 35 89 

2 18 46 

2 18 46 

-2 5 -25 

-19 -121 -422 

-10 -64 -209 

2 -2 24 

-20 -125 -430 

-5 -35 -118 



SOURCES: MDRC calculations from the State of Florida Unemployment Insurance (Ul) earnings and benefits records, and from published data on tax rates and 
employee fringe benefits. The end of the observation period was September 1993 for all outcome measures. 

NOTES: Estimates reflect discounting and adjustment for Inflation. 

Differences are regression -adjusted using ordinary least squares, controlling for pre-random assignment characteristics of sample members. 
Tests of statistical significance were not performed. 

Rounding may cause slight discrepancies in calculating sums and differences. 

(a) For each sample member, the projection base period estimate for a given outcome measure is the quarterly average of the estimates for the last 
four quarters of available follow-up. Program effects observed during this base period are multiplied by a projection factor to estimate benefits from the end of 

the observation period to the end of year 5. The projection factor for AFDC, Food Stamps, and Medicaid assumes that the effects of Project Independence observed 
in the base period decay at a rate of 25 percent over the remainder of the five-year period. The projection factor for earnings and related variables assumes that 
Project Independence's effects observed in the base period decay at a rate of 0 percent for sample members in the early cohort, and at a rate of 1 00 percent for 
sample members in the late cohort, over the remainder of the five-year period. ^ , 

(b) Florida does not have a state income tax. a* ' i i 

^ ■;• 
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TABLE 7.4 



PROJECTED FIVE-YEAR IMPACTS OF PROJECT INDEPENDENCE ON SELECTED OUTCOME MEASURES. 
PER PROGRAM GROUP MEMBER, ASSUMING ALTERNATIVE RATES OF DECAY 

(IN 1993 DOLLARS) 



Split Decay Rate, 

M§§mr.? . _ 0% Decay Rate 25% Decay Rate 45% Decay Rate by Cohort (a) 

Earnings 464 452 438 600 

AFDC -470 -422 -389 n/a 

Food Stamps -235 -209 -192 n/a 



SOURCES: MDRC calculations fronn the State of Florida Unemploynnent Insurance (U!) earnings and benefits records. The end of the observation 
period was September 1993 for all outcome measures. 

NOTES: Estimates reflect discounting and adjustment for inflation. 

Differences are regression -adjusted using ordinary least squares, controlling for pre-random assignment characteristics of sample members. 
Tests of statistical significance were not perfomned. 

Rounding may cause slight discrepancies in calculating sums and differences. 

(a) The split decay rate for earnings uses a projection factor that assumes the effects of Project Independence observed in the base period decay 
at a rate of 0 percent for sample members in the early cohort, and at a rate of 100 percent for sample members in the late cohort, over the remainder of 
the five-year period. "N/a" is used for AFDC and Food Stamps to indicate that the split decay rate is not applicable. 



in. Comparing Benefits and Costs for the Full Research Sample 



Table 7.5 summarizes Project Independence's monetary effects from the welfare sample, 
goverranent budget, taxpayer, and societal perspectives. The analysis defines program-control group 
differences as gains (indicated by positive values) and losses (indicated by negative values). Results 
are then added together to produce an estimate of the overall net gain or loss - net present value - 
of the Project Independence program from the perspective in question. As indicated earlier, all 
estimates for society as a whole constitute the sum of the results for the welfare sample and taxpayer 
perspectives (the latter includes the government budget perspective). All results cover a five-year 
period, are discounted and expressed in 1993 dollars, and reflect the assumption of a 25 percent decay 
in impacts on AFDC, Food Stamps, and Medicaid payments and a split rate of decay (0 percent for 
the early cohort and 100 percent for the late cohort) in impacts on earnings-related outcomes during 
the projection period. Five-year estimates of the net cost of providing Project Independence and non- 
Project Independence services are taken from Table 4.6. Because of the uncertainty surrounding the 
projections of cost and benefits beyond the observation period, this section of the chapter concludes 
by examining various benefit-cost results based on alternative assumptions about the projection 
estimates. 

A. Results from the Perspective of the Welfare Sample 

The first column of Table 7.5 presents the benefit-cost results from the perspective of the 
welfare sample. It should be remembered that these results represent program-control group 
differences in earnings and fringe benefits, taxes, and transfer payments. As discussed earlier, tlie 
overall net gain or loss from the perspective of the welfare sample is estimated by subtracting the 
combined value of tax increases and savings in transfer payments from the value of earnings gains and 
increased fringe benefits. As the first colunm of Table 7.5 shows, Project Independence program 
group members, on average, experienced a net financial loss as a result of the program, estimated at 
$369 per program group member over the five-year period. Tnis occurred because the increase in 
earnings produced by the program was not enough to compensate for the decreases in AFDC, Food 
Stamps, and Medicaid benefits. As discussed in Chapter 5, this was likely to have occurred because 
program group members typically left AFDC for a short-term, relatively low-wage job, but then did 
not return to the welfare rolls immediately (or at all) after leaving that job. 

B. Results from the Perspective of the Government Budget 

The second column of Table 7.5 presents the benefit-cost findings from the perspective of the 
government budget. On average. Project Independence produced a small net gain to the government 
budget of $72 per program group member. One can also consider the cost-effectiveness of the Project 
Independence program from the standpoint of the government budget by estimating the value of 
budgetary savings and tax increases per dollar of investment (i.e., per dollar of net costs of Project 
Independence and non-Project Independence employment-related activities and services). This measure 
is called return to budget per net dollar invested and is presented in the last row of Table 7.5. One 
calculates Project Independence's return to budget by adding together gains in taxes and savings in 
transfer payments and associated administrative costs and then dividing this total by the total net cost 
of services (both Project Independence and non-Project Independence). According to this measure, 
government budgets come out ahead if the program produces more than a dollar's worth of additional 
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TABLE 7.5 



FIVE-YEAR ESTIMATED NET GAINS AND LOSSES AND RETURN 
PER PROJECT INDEPENDENCE PROGRAM GROUP MEMBER. BY ACCOUNTING PERSPECTIVE 

(IN 1993 DOLLARS) 



Accounting Perspective 



Component of Analysis Welfare Sample 


Govemment 
Budget 


Taxpayer 


Society 


Earnings 


600 


0 


0 


600 


Fringe benefits (a) 


89 


0 


0 


89 


Tax payments 
Payroll taxes 

Income and sales taxes (b) 


-46 

25 


92 
-25 


46 

-25 


0 
0 


Transfer programs 
AFDC payments 
Food Stamps 
Ul compensation 
Total Medicaid 
Transfer administrative costs 


-422 
-209 
24 
-430 
0 


209 
-24 

118 


209 
-24 

118 


u 
0 
0 

U 

118 


Net cost of Project Independence and 
non-Project Independence 
activities and services 


0 


-1150 


-1150 


-1150 


Preference for work over welfare 


+ 


0 


+ 


•4- 


Foregone personal 

dilU iaiiiliy aOllVllKsb 




0 


0 




Value of education not 
reflected in earnings 


+ 


0 


+ 


+ 


Net gain or loss 
(net present value) 


-369 


72 


26 


-343 


Return to government budget per net dollar 
invested in Project Independence 
and non-Project Independence 
activities and services (c) 


n/a 


1 .06 per $1 


1.02 per $1 


n/a 



SOURCES: MDRC calculations from the State of Florida Unemployment Insurance (Ul) earnings and benefits records, 
and from published data on tax rates and employee fringe benefits. The end of the observation period was September 
1 993 for all outcome measures. 



NOTES: Results include estimates of projected program effects beyond the obsen/atlon period (see Table 7.3). 

(a) These include employer-paid health and life insurance, pension contributions, and workers' compensation. 

(b) Florida does not have a state income tax. 

(c) The return to government budget per net dollar Invested In Project Independence and non -Project 
Independence activities anci services Is computed by dividing tax payments and transfer program savings by the net cost 
of Project Independence and non -Project Independence activities and sen/Ices. ("N/a" means not applicable.) 
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revenues and savings for each additional dollar spent on employment-related services to program group 
members (compared to control group members). 

As seen in the second column of Table 7.5, Project Independence led to a return of $1.06 in 
revenues and savings for every net dollar spent on program group members (i.e., an increase of six 
cents per net dollar spent) - a marginally positive return to the budget which, given the uncertainties 
behind parts of the analysis, is probably best viewed as a "break-even" effect.^^ 

C. Results from the Perspective of Taxpayers 

The results from the taxpayer perspective are almost identical to those from the perspective of 
the government budget. As discussed earlier in this chapter, taxpayers (defined for this analysis as 
everyone in society other than the welfare sample) realized mostly the same benefits as the government 
budget. However, they benefited less from increased payroll taxes because only the employee share 
of those taxes is counted as a benefit to taxpayers. As the third column of Table 7.5. shows, the five- 
j^ear net gain for taxpayers was estimated to be $26, or about one-third the size of the gains for the 
government budget. 

Results from the Perspective of Society as a Whole 

The last column of Table 7.5 presents the final benefit-cost results from the societal perspective 
— a combination of the welfare sample and taxpayer perspectives. Once again, benefits accrued to 
society through earnings gains and savings in transfer program administration. All other effects 
represent gains from one perspective and losses from the other, resulting in no effect for society as 
a whole. The net present value of the program to society as a whole (the sum of taxpayer and welfare 
sample gains) was $343 below the break-even point. As previously discussed, this assumes that a 
dollar lost by one group has the same value as a dollar gained by another group, and that Project 
Independence caused no displacement effects. 

E. Comparing Results That Exclude and Include Projected Benefits or Costs 

As discussed earlier in this chapter, the reliance on two to three years of projected benefits and 
costs in estimating the program's results over a five-year time horizon introduces considerable 
uncertainty into the analysis. One way to gauge how much those projections affect the overall 
conclusions one would draw from the analysis is to compare the pattern of results based on estimates 
that include the projected values of benefits and costs with the pattern found when the projected values 
are excluded. The in-text table that follows offers such a comparison for the welfare sample and 
government budget perspectives. 



The ratio of $L06 returned to budget per net dollar invested is calculated by dividing the combined 
savings in transfer payments and administrative costs plus tax increases ($1,222) by the $1,150 in net costs 
of providing Project Independence and non-Project Independence activities and services to program group 
members. 
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Net Gain or Loss (Net Present Value) in 1993 Dollaxs 



Effect 



Welfare 
Sample 
Perspective 



Government 

Budget 
Perspective 



Observed estimates only (within 
two to three years after random 
assignment) 



-$304 



-$55 



Observed plus projected estimates 
(within five years after random 
assignment) 



-$369 



$72 



As the table shows, whether or not the projected benefits and costs are included in the analysis, 
the pattern of results is quite similar: The welfare sample experiences a net financial los:. exceeding 
$300 per person in the program group (reflecting the low full-sample earnings impacts), while the 
government budget roughly breaks even (i.e., a marginal loss is estimated without making projections 
and a marginal gain is estimated if projected values are included). 

rV, Benefit-Cost Results for Key Subgroups 

The final objective of the benefit-cost analysis is to determine whether the economic gains and 
losses produced by Project Independence varied for sample members according to the time they entered 
the program and the age of their youngest child. Comparing the early and late cohorts is a way to 
assess whether the program was more successful for thoFC who (for the most part) experienced it early 
on, when it was operated more as its designers had intended and in a more favorable economic 
climate. (See Chapter 1 for a discussion of these changes.) In addition, by separately examining 
within each cohort the results for the child-age subgroups, the analysis can determine whether being 
in the earlier versus the later cohort mattered more for sample members with preschool-age children 
or for those whose children were all at least 6 years old. 

Before reviewing the findings, it important to stress that the subgroup estimates are subject to 
greater uncertainty than are the full-sample results because the sample sizes on which the estimates 
were based are considerably smaller, and because a number of simplifying assumptions had to be made 
in performing the calculations (especially in estimating the variation in program costs by subgroup). 
Thus, the reader should focus more on the broad patterns of results than on the specific numerical 
estimates. 

With those caveats in mind, one can see in comparing the first two panels of Table 7.6 that 
there were substantial differences in benefits and costs for the early and late cohorts among sample 
members whose youngest child was age 6 or older. For example, the estimated five-year net 
government ''ost (for Project Independence and non-Project Independence activities and services) 
dropped from $1,128 for the early cohort to $877 for the late cohort, a reduction of $251, or 22 
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TABLE 7.6 



FIVE-YEAR ESTIMATED NET GAINS AND LOSSES AND RETURN 
PER PROJECT INDEPENDENCE PROGRAM GROUP MEMBER. BY ACCOUNTING PERSPECTIVE. 
BY RANDOM ASSIGNMENT COHORT AND AGE OF YOUNGEST CHILD 

(IN 1993 DOLLARS) 



Accounting Perspective 



Subgroup and Component 



Welfare Sample 



Government 
Budget 



Taxpayer 



Society 



Youngest child, age 6 or o lder: 

Early cohort 

Total from earnings 
and fringe benefits 

Total from tax payments (a) 

Total from transfer payments 
and administrative costs (b) 

Net cost of Project Independence 
and non -Project Independence 
activities and services 



1817 
-175 

--1309 



0 
296 

1454 
-1128 



0 
175 

1454 
-1128 



Net gain or loss 

(net present value) 333 621 500 

Return to government budget per net dollar 
invested in Project independence 
and non-Project Independence 

activities and services (c) n/a 1 .55 per $1 1 .44 per $ 1 



1817 
0 

145 

-1128 
834 



n/a 



657 
8 

-1420 



Youngest child, age 6 or older: 

Late cohort 

Total from earnings 
an'i fringe benefits 

■ Total from tax payments (a) 

Total from transfer payments 
and administrative costs (b) 

Net cost of Project Independence 
and Project Independence 
activities and services 



Net gain or loss 
(net present value) -755 

Return to government budget per net dollar 
invested in Project Independence 
and non-Project Independence 
activities and services (c) n/a 



0 
36 

1567 

-877 
745 



0 
-8 

1567 

-877 
702 



1 85 per$1 



1.80 per $1 



657 
0 

167 

-877 
-53 

n/a 



(continued) 
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TABLE 7.6 (continued) 



Accounting Perspective 



Benefit-Cost Component 



Welfare Sample 



Govemment 
Budget 



Taxpayer 



Society 



Youngest child, age 3-5: 

Early cohort 

Total from earnings 
and fringe benefits 782 

Total from tax payments (a) 50 

Total from transfer payments 
and administrative costs (b) -554 

Net cost of P roject I ndependence 
and non-Project Independence 
activities and services 0 

Net gain or loss 
(net present value) 277 

Return to government budget per net dollar 
invested in Project Independence 
and non-Project Independence 
activities and services (c) n/a 



0 
3 

609 

-1831 
-1220 

0.33 per $1 



0 

-50 
609 

-1831 
-1272 

0.31 per $1 



782 
0 

54 

-1831 
-995 

n/a 



Youngest child, age 3-5: 

Late cohort 

Total from earnings 
and fringe benefits -291 

Total from tax payments (a) 55 

Total from transfer payments 
and administrative costs (b) -512 

Net cost of Project Independence 
and non -Project Independence 
activities and services 0 

Net gain or loss 
(net present value) -748 

Return to government budget per net dollar 
invested In Project Independence 
and non-Project Independence 
activities and services (c) n/a 



0 

-74 
567 

-894 
-402 

0.55 per$1 



SOURCES: See Table 7.5. 



0 

-55 
567 

-894 
-382 

0.57 per $1 



-291 
0 

55 

-894 
-1130 

n/a 



NOTES: See Table 7.5. 

(a) This category includes payroll. Income, and sales taxes. Florida does not have a state Income tax. 

(b) This category Includes AFDC payments, Food Stamps, Unemployment Insurance compensation. 
Medicaid, and transfer administration costs. 

(c) The return to government budget per net dollar Invested in Project Independence and non-Project 
Independence activities and services Is computed by dividing tax payments and transfer program savings by the not cost 
of Project Independence and non-Project Independence activities and services. 
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percent. From the government budget perspective, this drop in net costs resulted in a greater 
overall economic gain: The return per net dollar invested rose from $1.55 to $L85. However, from 
the welfare sample perspective, being in the late cohort meant a financial loss of $755 per program 
group member over five years, compared to a gain of $333, on average, for those in the early cohort. 
Unlike the early cohort, the late cohort's earnings gains were simply too small to make up for their 
losses in transfer benefits (especially AFDC, Food Stamps, and Medicaid). 

The third and fourth panels of Table 7.6 present the findings for sample members with 
preschool-age children. Again, average net costs were lower for the late cohort - but by a much 
larger margin than was the case for the subgroup with older children. Estimated five-year net costs 
fell from $1,831 per program group member in the early cohort to $894 for those in the late cohort, 
a reduction of $937, or 51 percent. However, for neither of the cohorts did the government budget 
come out ahead for those with younger children. In fact, less than a dollar was returned to the budget 
for every net public dollar invested, largely because the combined savings in reduced tjansfer program 
costs were smaller than the net government outlays per program group member for employment-related 
activities and services. From the welfare sample perspective, however, the pattern of results was 
similar to that observed for the sample members whose children were all age 6 or older: For those 
with preschool-age children, being in the early cohort meant a small net gain of $277 on average over 
five years, while being in the late cohort resulted in a net loss of $748. 

Overall, then. Project Independence was most successful from a benefit-cost standpoint for the 
early cohort of sample members whose children were at least 6 years old. It achieved savings for the 
government budget, while at the same time making the welfare sample somewhat better off financially. 
Among the other three subgroups, either the welfare sample, the government budget, or both were 
made worse off financially. 



^^Overall, the lower net costs of serving the late versus the early cohort (for each of the child-age 
subgroups) are due in large part to a reduction in net expenditures on vocational training and post-secondary 
education. 

^^Project Independence child care costs, overall, do not account for a large share of the five-year estimated 
total gross cost or net cost per program group member with a preschool-age child. However, the reduction 
over time in access to child care may have affected those overall costs indirectly by making participation in 
relatively expensive vocational training and post-secondary education activities more difficult. Project 
Independence child care expenditures were estimated to be roughly $230 per program group member with 
preschool-age children in the early cohort (averaged across those who did and did not use child care). This 
average fell to only about $80 for those in the late cohort (a 65 percent decline). Also, subtracting imputed 
payments for control group members with preschool-age children (who could also receive Project 
Independence child care) reveals a lower net expenditure per program group member. One estimate shows 
those net expenditures falling from an average of roughly $80 for the early cohort to close to $60 for those 
in the late cohort (a 25 percent decline). 

Although none of these numbers is precise, the downward direction of the changes in both the gross 
and net cost estimates is consistent with program staff members* descriptions of their responses to the reduced 
availability of Project Independence child care over time. Staff reported that they typically either encouraged 
individuals who needed child care and were eligible to enter vocational training or post-secondary education 
to try job search activities instead or did not assign them to any activhies at all. 

It should also be noted that average Project Independence child care expenditures for sample members 
with children ages 6 and older were quite low to begin with (e.g., roughly $10 per program group member 
in the early cohort), and there was little difference across cohorts. 
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APPENDIX A 
SUPPLEMENTAL TABLES TO CHAPTER 2 



TABLE A.1 



SELECTED CHARACTERISTICS OF AFDC APPLICANTS 
AND OF APPLICANT SUBGROUPS DEFINED BY RESEARCH STATUS. 
AT THE TIME OF RANDOM ASSIGNMENT 



Subgroups, by 
Research Status 



Characteristic and Subgroup 



Sample 
Size 



All AFDC 
Applicants 



Program 
Group 



Control 
Group 



Ethnicity (%) 
White. non-Hispanic 
Black, non-Hispanic 
Hispanic 
aher 

Data not available 

Primary language (%) 
English 
Spanish 
Other 

Data not available 
Average age (years) 

Number of children (%) 

1 child 

2 children 

3 or more children 
Data not available 

Age of youngest child (%) 
Age 3 to 5 
Age 6 or older 
Data not available 

Total prior AFDC receipt (a) (%) 
First-time applicant 

Applicant with less than 2 years of AFDC receipt 
Applicant with 2 yea^s or more of AFDC receipt 
Data not available 

Education (%) 
High school diploma or GED 
No high school diploma or GED 
Data not available 

Any earnings during the prior year (%) 

Job- readiness status (b) {%) 
Job-ready 
Not job -ready 
Data not available 

Random assignment cohort (%) 
Early cohort (July-December 1990) 
Late cohort (January-August 1991) 



5,583 
5,755 
3,639 
180 
832 



12,168 
2,893 
264 
664 

15,989 



7.087 
4,921 
3,112 
869 



5.958 
8.607 
;.424 



7,120 
4.373 
3.593 
903 



8,499 
6.508 
982 

15,089 



13.328 
1,539 
1,122 



6.705 
9.284 



34.9 
36.0 
22.8 
1.1 
5.2 



76.1 
18.1 
1.7 
4.2 

32.3 



44.3 
30.8 
19.5 
5.4 



37.3 
53.8 
8.9 



44.5 
27.4 
22.5 
5.7 



53.2 
40.7 
6.1 

60.8 



83.4 
9.6 
7.0 



41.9 
58.1 



35.1 
36.1 
22.7 
1.1 
5.1 

76.3 
18.1 
1.5 
4.1 

32.3 

44.6 
30.4 
19.7 
5.3 



37.3 
53.9 
8.9 



44.5 
27.5 
22.5 
5.5 



53.3 
40.8 
5.9 

60.5 



83.5 
9.7 
6.8 



41.9 
58.1 



34.5 
35.8 
23.0 
1.3 
5.5 



75.5 
18.2 
2.0 
4.3 

32.2 



43.4 
32.0 
18.8 
5.9 



37.3 
53.7 
9.1 



44.7 
26.8 
22.4 
6.1 



52.9 
40.3 
6.9 

61.9 



83.0 
9.5 
7.5 



42.1 
57.9 



Sample size 



15,989 



1 1 .946 



4.043 



(continued) 
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TABLE A. 1 (continued) 



SOURCES: MDRC calculations from Background Information Forms and Florida Unemployment Insurance (Ul) records. 

NOTES: Distributions may not add to 100.0 percent because of rounding. 

A chi-square test or two-tailed t-test was applied to differences between subgroups. Statistical 
significance levels are indicated as *** ~ 1 percent; ** = 5 percent; * = 10 percent. 

(a) This refers to the total number of months accumulated from one or more spells on an individual's own or 
spouse's AFDC case. It does not include AFDC receipt under a parent's name. 

(b) Sample members are defined as "job-ready" if they had completed at least 10th grade or were employed 
for at least 1 2 of the 36 months prior to random assignment. They are defined as "not job-ready" rf they did not meet both 
Ci iteria. These definitions are based on those used by Project Independence during the the random assignment period. 
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TABLE A.2 



SELECTED CHARACTERISTICS OF AFDC RECIPIENTS 
AND OF RECIPIENT SUBGROUPS DEFINED BY RESEARCH STATUS. 
AT THE TIME OF RANDOM ASSIGNMENT 



Characteristic and Subgroup 



Ethnicity (%) 
White, non-Hispanic 
Black, non-Hispanic 
Hispanic 
aher 

Data not available 

Primary language (%) 
English 
Spanish 
aher 

Data not available 
Average age (years) 

Number of children (%) 

1 child 

2 children 

3 or more children 
Data not available 

Age of youngest child (%) 
Age 3 to 5 
Age 6 or older 
Data not available 

Total prior AFDC receipt (a) (%) 
Recipients with less than 2 years of AFDC receipt 
Recipients with 2 years or more of AFDC receipt 
Data not available 

Education (%) 
High school diploma or GED 
No high school diploma or GED 
Data not available 

Any earnings during the prior year (%) 

Job-readiness status (b) (%) 
Job -ready 
Not job -ready 
Data not available 

Random assignment cohort (%) 
Early cohort (July-December 1990) 
Late cohort (January-August 1991) 



Sample 
Size 



All AFDC 
Recipients 



535 
1,123 
405 
27 
158 



1,719 
339 
39 
151 

2.248 



773 
654 
685 
136 



1.253 
709 
286 



790 
1.029 
429 



938 
1.035 
275 

2.248 



1.608 
277 
363 



806 
1.442 



23.8 
50,0 
18.0 
1.2 
7.0 



76.5 
15.1 
1.7 
6.7 

30.5 



34.4 
29.1 
30.5 
6.1 



55.7 
31 .5 
12.7 



35.1 
45.8 
19.1 



41.7 
46.0 
12.2 

63.0 



71.5 
12.3 
16.2 



35.9 
64.2 



Subgroups, by 
Research Status 



Program 
Group 



23.6 
49.8 
18.1 

1.? 

7.3 



76.0 
15,4 
1.6 
7.0 

30.5 



34.6 
29.8 
29.7 
5.9 



55.5 
31.5 
13.1 



35.9 
44.0 
20.2 



42.8 
45.1 
12.2 

63.4 



72.2 
12.3 
15.6 



39.1 
60.9 



Control 
Group 



Sample size 



2,248 



1.567 



24.4 
50.4 
17,9 
1.0 
6.3 



77.5 
14.4 
2.1 
6.0 

30.5 



33.9 
27.5 
32.2 
6.5 



56.4 
31.7 
11.9 



33.5 
49.9 *** 
16.6 ** 



39.4 
48.3 
12.3 

62.0 



70.0 
12.5 
17.5 



28,5 *** 
71.5 *** 

681 



SOURCES AND NOTES: See Table A.I . 
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TABLE A.3 



SELECTED CHARACTERISTICS OF THE FULL SAMPLE AND 
OF SUBGROUPS DEFINED BY AGE OF YOUNGEST CHILD, 
AT THE TIME OF RANDOM ASSIGNMENT 



Sample 
Size 



Characteristic and Subgroup 

Ethnicity (%) 

White, non-Hispanic 6,1 18 

Black, non-Hispanic 6,878 

Hispanic 4,044 

Other 207 

Data not available 990 

Primary language (%) 

English 13,887 

Spanish 3,232 

Other 303 

Data not available 81 5 

Average age (years) 1 8,237 

Number of children (%) 

1 child 7,860 

2 children 5,575 

3 or more children 3,797 
Data not available 1 ,005 

Age of youngest child (%) 

Age 3 to 5 7,211 

Age 6 or older 9,316 

Data not available 1 ,71 0 

Total prior AFDC receipt (a) (%) 

First-time applicant 7,1 20 

Applicant with less than 2 years of AFDC receipt 4,373 

Recipient with less than 2 years of AFDC receipt 790 

Applicant with 2 years or more of AFDC receipt 3,593 

Recipient with 2 years or more of AFDC receipt 1 ,029 

Data not available 1 ,332 

Education {%) 

High school diploma or GED 9,437 

No high school diploma or GED 7,543 

Data not available 1 ,257 

Any earnings during the prior year (%) 1 8,237 

Job-readiness status (b) (%) 

Job -ready 14,936 

Not job- ready 1,816 

Data not available 1 ,485 

Research sample status (%) 

Program group 1 3,51 3 

Control group 4,724 

Random assignment cohort (%) 

Early cohort (July-December 1 990) 7.51 1 

Late cohort (January -August 1 991) 10.726 



Full 
Sample 



Subgroups, by Age of 
Youngest Child 



Age 6 or Older 



Age 3 to 5 



33.5 
37.7 
22.2 
1.1 
5.4 

76.1 
17.7 
1.7 
4.5 

32.1 



43.1 
30.6 
20.8 
5.5 

39.5 
51.1 
9.4 



39.0 
24.0 
4.3 
19.7 
5.6 
7.3 



51.7 
41.4 
6.9 

61.1 



81.9 
10.0 
8.1 



74.1 
25.9 



41.2 
58.8 



34.9 

37.3 *** 
24.8 *** 

1.2 

1.9 

77.1 *** 
20.3 *** 

1.7 

0.9 

35.0 *** 



49.7 *** 
32.5 

17.8 *** 
0.0 

0.0 
100.0 
0.0 



42.9 
24.9 
2.7 
22.3 
3.2 
4.0 



54.3 ** 
42.8 
2.9 *** 

61.1 



85.8 *** 
10.2 

4 Q *** 

74.4 
25.6 

40.9 *** 
59.1 *** 



35.4 
40.6 
20.7 
1.2 
2.2 



81.6 
15.8 
1.7 
1.0 

28.6 



38.9 
33.2 
27.9 
0.0 



100.0 
0.0 
0.0 



36.7 
25.3 
6.2 
18.9 
9.0 
3.9 

52.8 
43.4 
3.9 

60.9 



84.0 
10.6 
5.4 

73.8 
26.2 



43.6 
56.4 



Sample size 



18.237 



9.316 



7.211 



SOURCES: See Table A.I. 



NOTES: See Table A.I. 

Sample sizes for subgroups do not add to the full sample size because of missing data. Therefore, the weighted 
average of the percentages of the subgroups wrthin a characteristic does not equal the percentage for the full sample. 
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TABLE A.4 



SELECTED CHARACTERISTICS OF THE FULL SAMPLE AND 
OF SUBGROUPS DEFINED BY JOB-READINESS STATUS. 
AT THE TIME OF RANDOM ASSIGNMENT 



Characteristic and Subgroup 



Subgroups, by 
Sample Full Job~Readiness Status 

Size Sample Job-Ready Not Job-Ready 



Ethnicity (%) 

White. non-Hispanic 6,1 18 

Black, non- Hispanic 6,878 

Hispanic 4,044 

Other 207 

Data not available 990 

Primary language (%) 

English 13,887 

Spanish 3,232 

Other 303 

Data not available 815 

Average age (years) 18.237 

Number of children (%) 

1 child 7.860 

2 children 5,575 

3 or more children 3,797 
Data not available 1 .005 

Age of youngest child (%) 

Age 3 to 5 7,21 1 

Age 6 or older 9.316 

Data not available 1 ,71 0 

Total prior AFDC receipt (a) (%) 

First-time applicant 7,120 

Applicant with less than 2 years of AFDC receipt 4,373 

Recipient with less than 2 years of AFDC receipt 790 

Applicant with 2 years or more of AFDC receipt 3,593 

Recipient with 2 years or more of AFDC receipt 1 ,029 

Data not available 1 ,332 

Education (%) 

High school diploma or QED 9,437 

No high school diploma or GED 7,543 

Data not available 1 ,257 

Any earnings during the prior year (%) 1 8,237 

Job -readiness status (b) (%) 

Job-ready 14,936 

Not job -ready 1,816 

Data not available 1 .485 

Research sample status (%) 

Program group 1 3,51 3 

Control group 4 .724 

Random assignment cohort (%) 

Early cohort (July-December 1990) 7,51 1 

Late cohort (January-August 1991) 1 0,726 



33.5 
37.7 
22.2 
1.1 
5.4 



76.1 
17.7 
1.7 
4.5 

32.1 



43.1 
'30.6 
20.8 
5.5 



39.5 
51.1 
9.4 



39.0 
24.0 
4.3 
19.7 
5.6 
7.3 

51.7 
41.4 
6.9 

61.1 



81.9 
10.0 
8.1 



74.1 
25.9 

41.2 
58.8 



33.7 
39.5 
20.0 
1.0 
5.7 



78.1 
15.6 
1.4 
4.9 

32.1 



43.2 
30.5 
20.3 
2.0 



39.3 
50.9 
9.8 



39.5 
23.9 
4.2 
19.2 
5.5 
7.7 

57.5 
34.9 
7.7 

61.2 



100.0 
0.0 
0.0 



74.1 
25.9 

40.7 
59.3 



32.1 
21.6 
41.6 
2.1 
2.6 



*** 
*** 
*** 
*** 



58.5 *** 
37.2 *** 
3.6 *** 
0.7 *** 

32.8 *** 



42.0 
30.8 
25.2 *** 
5.9 *** 

42.0 ** 
52.5 
5.5 *** 



34.6 *** 
24.3 

5.2 * 
24.5 *** 

7.3 *** 
4.1 *** 



0.0 *** 
100.0 *** 
0.0 *** 

60.2 



0.0 *** 
100.0 *** 
0.0 



74.3 
25.7 

45.3 *** 
54.7 *** 



Sample size 



18.237 



14,983 



1,816 



SOURCES AND NOTES: See Tables A.I and A.3. 
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TABLE A.5 



SELECTED CHARACTERISTICS OF THE FULL SAMPLE AND 
OF SUBGROUPS DEFINED BY PRIOR AFDC RECEIPT. 
AT THE TIME OF RANDOM ASSIGNMENT 



Subgroups, by Prior AFDC Receipt (a) 
Sample Full Less Than 2 or More 

Characteristic and Subgroup Size Sample None 2 Years Years 

Ethnicity (%) 



White, non-Hispanic 


6,118 


33.5 


39.9 


36.1 


26.5 *** 


Black, non— Hispanic 


6,878 


37.7 


26.7 


41.4 


55.8 *** 




4,044 




29.7 


19.7 


15.6 *** 


other 


207 


1 .1 


1 ft 


0 8 


Q g *** 


Data not available 


990 


5.4 


1.9 


2.0 


1^5 


Primary language (%) 












English 


13,887 


76.1 


72.8 


82.5 


86.1 *** 


Spanish 


3,232 


17.7 


24.4 


15.1 


11.8 *** 


Other 


303 


1 .7 


2.2 


1.5 


1 .3 *** 


Data not available 


815 


4.5 


0.7 


0.9 


0.8 


Average age (years) 


18,237 


32.1 


32.9 


31.1 


31.7 *** 


Number of children (%) 












1 child 


7,860 


43.1 


51.4 


44.6 


34.9 


2 children 


5,575 


30.6 


30.4 


32.9 


32.3 *** 


3 or more children 


3,797 


20.8 


15.8 


20.6 


31.5 *** 


Data not available 


1 ,005 


5.5 


2.5 


1,9 


1.3 *** 


Age of youngest child (%) 












Age 3 to 5 


7,211 


39.5 


37.1 


44.0 


43.6 *** 


Age 6 or older 


9,316 


51.1 


56.2 


49.9 


51 .5 *** 


Data not available 


1,710 


9.4 


6.7 


6.1 


4 g *** 


Total prior AFDC receipt (a) (%) 












First— time applicant 


7.120 


39.0 


100.0 


0.0 


0.0 *** 


Applicant with less than 2 years of AFDC receipt 


4,373 


24.0 


0.0 


84.7 


0.0 


Recipient with less than 2 years of AFDC receipt 


790 


4.3 


0.0 


15.3 


0.0 *** 


Applicant with 2 years or more of AFDC receipt 


3,593 


19.7 


0.0 


0.0 


77.7 *** 


Recipient with 2 years or more of AFDC receipt 


1 ,U^b} 


O.D 


U.U 


U.U 




Data not available 


1 ,oo^ 


"7 Q 
/ .O 


o o 

U.U 


U.U 


U.U 


Education (%) 












High school diploma or GED 


9,437 


51.7 


59.3 


53.8 


46.0 *** 


No high school diploma or GED 


7,543 


41.4 


37.3 


43.3 


50.5 *** 


Data not available 


1,257 


6.9 


3.4 


2.9 


3.6 


Any earnings during the prior year (%) 


18,237 


61.1 


61.2 


61.2 


60.5 


Job-readiness status (b) (%) 












Job- ready 


14.936 


81.9 


87.3 


85.1 


82.3 *** 


Not job -ready 


1,816 


10.0 


8.8 


10.4 


12.5 *** 


Data not available 


1.485 


8.1 


3,9 


4.5 


5.2 *** 


Research sample status (%) 












Program group 


13,513 


74.1 


74.6 


74.6 


73.0 


Control group 


4.724 


25.9 


25.4 


25.4 


27.0 


Random assignment cohort (%) 












Early cohort (July-December 1990) 


7,511 


41.2 


41 .3 


42.1 


42.1 


Late cohort (January-August 1991) 


10.726 


58.8 


58.7 


57.9 


57.3 


Sample size 




18,237 


7,120 


5.163 


4.622 



SOURCES AND NOTES: See Tables A.I and A.3. 
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APPENDIX B 



RESULTS OF THE 24-MONTH SURVEY RESPONSE ANALYSIS 



Data on a number of participation measures and on job quality and other outcomes came 
primarily from the 24-mbnth Project Independence survey. This survey was targeted to a particular 
subsample of the full research sample of program and control group members, as discussed in Chapter 
2. The survey sample was designed to be representative, but not all research sample members were 
in the pool from which the survey sample was drawn. In addition, not all sample members selected 
to be surveyed could be interviewed. Some could not be located, and some refused or were unable 
to be interviewed. Sample members who completed the survey are called respondents; collectively, 
they are the survey respondent sample. Sample members selected for the survey who did not complete 
it are called nonrespondents or the survey nonrespondent sample. The sample of respondents plus non- 
respondents is the survey sample. In Project Independence, respondents made up more than four-fifths 
of the survey sample. The survey sample plus those sample members who were not selected to be 
surveyed is called the full research sample. 

Whenever survey response rates are less than 100 percent, two kinds of biases may be present. 
First, it is possible that respondents in the program group may have different characteristics from 
control group respondents. In that case, the fundamental comparison between program and control 
group members may be invalid, and impact estimates based on survey data may be biased. Second, 
the sample of completed surveys may not well represent the full survey sample or the full research 
sample. In that case, impact estimates for survey respondents may not generalize to all program 
enrollees. This appendix reports the results of an analysis of survey response patterns undertaken to 
determine whether either of those biases exists and is significant. Such an analysis is routinely 
performed in field studies using survey data. ^ 

To summarize the results presented below: It was found that there was little likelihood of 
substantial bias in the basic impact comparison between program and control group members within 
the respondent subsample. In addition, the respondent sample appears to represent the full research 
sample reasonably well. 

The total rate of response within the survey sample was 82 percent, exceeding the target rate 
of 80 percent. This response rate is high enough to suggest that the survey probably represents the 
full research sample quite well. Within the program group, the response rate was 83; among controls, 
it was 81. The similarity of these rates suggests the absence of significant biases in the basic impact 
comparison. 

To look more closely for any program-control differences within the survey respondent sample, 
the 0/1 dummy variable indicating membership in the program group was regressed on pre-random 



The issue of item nonresponse, i.e., the failure to respond to a particular question or set of questions, 
is not examined here. In most instances, item nonresponse was fairly low for sample members who otherwise 
responded to the survey. Information about item nonresponse is provided in tlie notes to tables and in the text 
where appropriate. 
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assignment demographic characteristics'using the survey respondent sample. The regression R-square 
was not statistically significant, which indicates that membership in one or the other research group 
was not related to pre-random assignment characteristics. This result, in conjunction with the high and 
similar response rates, supports the conclusion that the fundamental comparison of program and control 
groups among survey respondents should not have produced biased impact estimates. 

To help assess the generalizability of survey-based estimates, the 0/1 dummy variable indicating 
survey response versus survey nonresponse was regressed on pre-random assignment demographic 
information using the survey sample of respondents and nonrespondents combined. After controlling 
for research group and county,^ the increment in regression R-square was still statistically 
significant.-^ Among other differences, slightly higher response rates were obtained for survey sample 
members who were job ready under the revised definition, and slightly lower response rates were 
obtained for individuals who were never married. These (and other) differences were not large, 
however. The respondent sample should therefore be considered reasonably representative of the full 
research sample. 

Impact estimates for the Project Independence 24-month survey respondents were quite similar 
to those for the rest of the full research sample. Two-year earnings impacts for survey respondents 
were only about $20 larger than for the rest of the ftill research sample, a very small and not 
statistically significant difference. Two-year AFDC impacts were $19 larger, also a small and not 
statistically significant difference."^ Thus, other impacts estimated from survey data alone should 
provide reasonably valid estimates of beliavior in the full research sample. 



^Response was expected to differ across research group and county because the probability of selection 
into the survey sample was different for groups defined by those factors. The differences in probability of 
selection do not make the selected sample nonrandom. 

•^The incremental R-square was also statistically significant when the same regression was run on the full 
research sample. 

"^Impacts for survey respondents were not statistically significantly different from those of survey nonre- 
spondents, although die numerical estimates were quite different. Those numerical differences do not reflect 
on the generalizability of the survey; rather, they are a reflection of the small number of nonrespondents, 
which permits only imprecise impact estimates for them. 
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APPENDIX C 



ESTIMATING RATES OF PARTICIPATION IN 
PROJECT INDEPENDENCE ACTIVITIES AMONG CONTROL GROUP MEMBERS 



Preliminary analysis of the 24-month survey data indicated that, contrary to the research 
design, 44.8 percent of control group members responded to questions in a way that could be 
interpreted as meaning they were exposed to Project Independence activities during the two years 
following their random assignment date. This apparent violation of the research design prompted an 
intensive investigation of the accuracy of these self-reported data. Although a significant number of 
control group members were, in fact, exposed to the program, data collected from Project 
Independence casefiles indicated that the 24-month survey data substantially exaggerated the extent to 
which this occurred. In all, data collected from Project Independence casefiles indicated that an 
estimated 19.8 percent of control group members were exposed to some aspect of the Project 
Independence treatment. This is the most reliable estimate of the control group's involvement with 
the program and is the one that was adopted for this report. 

This appendix provides a description of the data collection and analysis procedures used to 
arrive at the estimate of the contro) group's exposure to Project Independence presented in Chapter 
3, The first part of the appendix presents findings taken directly from the 24-month survey on control 
group members' self-reported rates of involvement with Project Independence. The second part of the 
appendix describes findings from Project Independence casefile data that were collected to verify the 
accuracy of the self-reported survey information. 

Before beginning this discussion, it is important to note that the 24-month survey data appear 
to provide accurate estimates of the rate at which program group members participated in Project 
Independence activities. This conclusion is based on a comparison of Project Independence 
participation rates calculated from the 24-month survey data and participation rates calculated for the 
12-month program flow analysis, which utilized Project Independence casefile data collected within 
12 months following an individual's random assignment date. For example, both data sources 
indicated that approximately 32 percent of the program group members participated in Project 
Independence independent job search or job club activities during the first 12 months following random 
assignment. Both the 24-month survey data and the 12-month program flow analysis also indicated 
that approximately 13 percent of the program group members participated in Project Independence 
education or training activities within 12 months after random assignment. 

In addition, there is not sufficient evidence to conclude that the 24-month survey data provide 
inaccurate estimates of the control group's overall participation in employment-related activities 
(combining Project Independence and non-Project Independence activities). Thus, the primary 
conclusion of the analysis presented in this appendix is that control group members did not accurately 
label their participation in employment-related activities as part of Project Independence or not part 
of Project Independence. 
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L Control Group Participation in Project Independence Emplovment-Related Activities; 
Evidence from the 24-Month Survey 

According to the 24-month survey, 21.8 percent of the control group members reported 
participating in at least one employment-related activity (i.e., independent job search, job club, 
education, or training) through Project Independence. Some others who did not participate in any 
employment-related activities reported some exposure to Project Independence: 14.0 percent of control 
group members reported attending Project Independence orientation, and 1.0 percent reported that they 
were sanctioned (i.e., their AFDC grant was reduced) for not attending an employment or training 
program. Finally, an additional 8.1 percent of control group members reported that they were 
informed that they were "required to go to an education, training, or employment program in order 
to receive AFDC." Again, however, none of this last group of control group members reported 
participating in any Project Independence activities. The combination of all these mutually exclusive 
percentages - 21.8 plus 14.0 plus 1.0 plus 8.1 — yields the estimate of up to 44.8 percent of control 
group members who may have been exposed to some aspect of the Project Independence treatment.^ 
As discussed below, the ^-eview of i rojcct Independence casefiles indicated that this estimate is almost 
certainly much too hign. 

There are several likely reasons that control group members may have over-reported their 
involvement in Project Independence on the 24-month survey. First, many of the programs and 
organizations through which control group members obtained employment-related services are affiliated 
with Project Independence and often have particular classes and activities set aside for AFDC 
recipients. If the connection between these agencies and Project Independence was well advertised, 
it may be that some control group members attributed to Project Independence their participation in 
activities provided by these organizations, even though they were not referred there by the program 
and their participation was not paid for by Project Independence. Another explanation is that some 
control group members may have been involved in Project Independence as part of a previous spell 
on AFDC that occurred prior to the start of the evaluation. They may have mistakenly attributed this 
involvement to the evaluation period. 

Finally, control group members may have been subject to employment or job search 
requirements that were associated with their receipt of Food Stamps or Unemployment Insurance 
benefits. They may have attributed these requirements to Project Independence. In fact, Florida's 
Food Stamp Employment and Training Program is also called Project Independence and is operated 
out of the same offices as the AFDC Project Independence program. This version of Project 
Independence is targeted to Food Stamp recipients who are exempt from the AFDC requirement to 
attend tlie program (e.g., because they no longer receive AFDC) and who have no children under age 
six.^ The Food Stamp Project Independence program, although provided with very little funding and 
with few participants, consists primarily of an orientation session and an eight-week independent job 
search in which participants are required to submit job applications to 24 employers who have available 
positions. 



^These percentages add to 44.9 because of rounding, but 44.8 is the exact sum. 
^The Project Independence exemption criteria are listed in Chapter 1 , footnote 1 . 
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Recipients of Unemployment Insurance benefits are asked to meet job search requirements in 
order to continue receiving those benefits. An important distinction between Project Independence and 
the Food Stamp Employment and Training Program and the Unemployment Insurance job search 
program, however, appears to be that Project Independence was more rigorous in enforcing its 
participation mandate than either of the other two programs. As a result, control group members' 
participation in Project Independence may have been qualitatively different from their participation in 
other programs, even if they were involved in the same services. 

In short, the estimates from 24-month survey data represent an upper bound on the actual 
Project Independence participation rates among control group members. However, as discussed below, 
data collected from Project Independencr casefiles indicate that their actual rate of participation in the 
program was probably much lower. 

IL Control Group Participation in Project Independence Employment-Related Activities: 
Evidence from Project Independence Casefiles 

A Project Independence casefile is usually created for any individual who is referred to the 
program. The casefile contains a record of the case manager's contact with the individual and of the 
individual's participation in activities, use of support services, referral for sanctions, and employment. 
For the Project Independence evaluation, some program offices created casefiles for control group 
members — even though the control group members were not (or should not have been) referred to 
the program — as part of procedures to prevent them from participating in the program should they 
be referred inadvertently. 

A. Project Independence Casefile Data for the 24-Month Survey Verification Analysis 

To verify the accuracy of the 24-month survey information about participation in Project 
Independence, MDRC worked closely with staff in the local Project Independence units to locate 
casefiles for random subsamples of program and control group members who responded to the 24- 
month survey. Specifically, Project Independence staff were asked to try to locate casefiles for a 
random subsample of 250 control group members and 150 program group members.-^ 



^The subsample of 150 program group members was selected randomly from the 520 program group 
members who responded to the 24-month survey. The subsample of 250 control group members was designed 
to over-represent those who reported on the 24-month survey that they were exposed to Project Independence. 
Specifically, the control group subsample was drawn from four sets of 24-month survey respondents that were 
defined by their self-reported participation in employment-related activities: (1) 100 were randomly selected 
from the 1 11 control group members who reported participating in Project Independence employment-related 
activities; (2) 50 were randomly selected from the 53 control group members who reported participating in 
a Project Independence orientation, but not in employment-related activities; (3) 50 were randomly selected 
from the 93 control group members who reported participating in employment-related activities, but not 
through Project Independence; and (4) 50 were randomly selected from the 252 control group members who 
did not report participating in any employment-related activities. This strategy was used to increase the 
accuracy of participation measures for flie first two groups, who reported that they did participate in Project 
Independence. All estimates of participation for control group members were weighted to reflect the 
distribution of the casefile sample in the original 24-month survey sample. 
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In all. Project Independence casefiles were located for a total of 41 .8 percent of the 250 control 
group members who had been selected for this analysis and 77.4 percent of the 150 selected program 
group members. It was expected that many program group members and the majority of control group 
members would not have Project Independence casefiles. For example, as discussed in Chapter 3, 
some program group members never reached Project Independence because they were never approved 
for AFDC or they obtained employment shortly after being approved and before they were scheduled 
to attend a program orientation. Since control group members were supposed to be told that they were 
not allowed to participate in Project Independence, it is not surprising that the percentage of casefiles 
found for them was much lower than for the program group. At the same time, it was to be expected 
that some control group members would have Project Independence casefiles, even though they did 
not participate in the program during the ^o-year follow-up period. Some control group members 
had casefiles because they had been in the program in the past, long before the study began. Project 
Independence staff may have created casefiles for other control group members in order to store a 
record that they were part of the study's control group and were not supposed to be served. 

There were some program and control group members for whom casefiles were not found but 
who may have been involved in the program. Project Independence staff who conducted the casefile 
data collection for the 24-month survey verification analysis reported that they were aware of some 
program and control group members who had been involved in Project Independence, but whose 
casefiles were no longer accessible, usually because those casefiles had been archived or transferred 
to other locations. Unfortunately, it was not possible to obtain a precise count of the sample members 
who were involved in Project Independence but did not have a casefile at the time the 24-month survey 
verification data were collected."* Instead, for this study, the amount of missing casefile data was 
estimated based on findings for the program group, following the procedures discussed in the next 
section. 

B. Accounting for Missing Casefile Data 

As discussed above, findings from casefile data collected for the 12-month program flow 
analysis indicate that the 24-month survey data provide an accurate estimate of the actual rates of 
participation in Project Independence activities for program group members. Thus, any differences 
between tht; 24-month survey estimates and the estimates obtained from the Project Independence 
casefile data collected for the 24-month survey verification analysis are likely to represent the amount 
of missing casefile information for the program group. For example, the 24-month survey data 
indicate that 44.4 percent of the program group members participated in at least one Project 
Independence employment- related activity. The casefile data collected for the 24-month survey 
verification analysis, however, indicate that only 36.8 percent of the program group members 
participated in employment-related activities through Project Independence. This means that the 
casefile data reflect only about 82.9 percent of the actual rate of participation in Project Independence 
activities among program group members.^ 



"^The casefile data for the 24-month survey verification analysis were collected in August 1994. This was 
between three and four years after members of the reseaich sample were randomly assigned. Thus, it not 
surprising that some of the casefiles were missing owing to archiving or transfer. 

^Some of this difference may be due to the fact that casefile data were collected for only a subsample of 
the program group members who responded to the survey. 
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Similarly, the 24-month sxirvey data indicate that 37.5 percent of the program group members 
participated in Project Independence independent job search or job club activities. The casefile data 
collected for the 24-month survey verification analysis, however, indicate that only 28.2 percent of the 
program group members participated in independent job search or job club activities through Project 
Independence. This means that the casefile data reflect only about 75.2 percent of the actual rate of 
participation in independent job search or job club activities among program group members. 

Finally, the 24-month survey indicates that 16.4 percent of the program group members 
participated in Project Independence education or training activities. The casefile data for the 24- 
month survey verification analysis indicate, however, that 14.0 percent of the program group members 
participated in Project Independence education or training activities. This means that the casefile data 
reflect only about 85.4 percent of the actual rate of participation in education and training among 
program group members. 

As described below, these findings on missing casefile data for program group members were 
used to correct Project Independence participation estimates for control group members in order to 
account for missing casefile data for the control group. For example, the casefile data collected for 
the 24-month survey verification analysis indicate that 6.2 percent of the control group members 
participated in at least one Project Independence employment-related activity during the two-year 
follow-up period. Based on the program group findings discussed above, this estimate was assumed 
to represent 82.9 percent of the control group's actual rate of participation in these activities. As a 
result, to account for missing casefile data, this estimate was increased to 7.5 percent (i.e., 6.2 divided 
by .829). 

Similarly, the Project Independence casefile data indicate that 4.5 percent of the control group 
members participated in Project Independence independent job search or job club activities. Based on 
the program group findings discussed above, this estimate was assumed to represent 75.2 percent of 
the control group's actual rate of participation in independent job search or job club activities. As a 
result, to account for missing casefile data, this estimate was increased to 6.0 percent (i.e., 4.5 divided 
by .752). 

Finally, the casefile data collected for the 24-month survey verification analysis indicate that 
1 .9 percent of the control group members participated in Project Independence education and training 
activities. Based on the program group findings discussed above, this estimate was assumed to 
represent 85.4 percent of the control group's actual rate of participation in education or training 
activities. As a result, to account for missing casefile data, this estimate was increased to 2.2 percent 
(i.e., 1.9 divided by .854). 

C. Corrected Rates of Participation in Specific Project Independence Activities for 
Control Group Members 

The casefile data collected for the 24-month survey verification analysis did not include 
separate information on participation in specific Project Independence activities such as independent 
job search, job club, ABE or GED, ESL, vocational training and post-secondary education, or on-the- 
job training. Rates of participation in these individual activities, therefore, were estimated from the 
24-month survey, but were corrected using the casefile data for the aggregate categories of activities, 
namely, "independent job search or job club" and "education or training. " For example, the 24-month 
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survey data indicate that 9.4 percent of th-*^ control group members participated in education or training 
activities tlirough Project Independence. The casefile data (including the correction to account for 
missing casefiles), however, indicate that the control group's actual rate of participation in these 
activities was 2.2 percent, 77 percent lower than the 24~month survey estimate (i.e., 1 minus 2.2/9.4). 
Therefore, the separate rates of participation in education (ABE or GED, and ESL) and training 
(vocational training or post-secondary education, and on-the-job training) were also assumed to be 77 
percent lower than the rates indicated by the 24-month survey. The same metliod was used to estimate 
rates of participation in independent job search and job club activities. The results of these corrections 
are presented in Table 3.4 and reflect the most accurate estimates of control group participation in 
Project Independence employment-related activities. 

III. Corrected Rates of Participation in a Project Indepeadence Orientation and Referrals for 
Sanction 

The 24-month survey data indicate that 71.2 percent of program group members attended a 
Project Independence orientation. However, evidence from the 12-month program flow analysis gives 
this rate as 77 percent. Since the program flow analysis was prepared using Project Independence 
casefiles collected within 12 months after program group members were referred to the program, it 
is likely to be a more accurate estimate of the actual rate of participation in orientation. 

According to tlie casefile data collected later for the 24-month survey verification analysis, only 
47.5 percent of program group members attended a Project Independence orientation. Assuming that 
the 12-month program flow analysis findings are more complete implies that the casefile data for the 
24-month survey verification analysis represent oa^y 61.7 percent of the total rate of orientation 
attendance among program group members. This was used to correct the orientation attendance rate 
for control group members that was found in the casefile data. 

According to the casefile data, 11.5 percent of control group members were found to have 
attended orientation. Assuming that this represents 61,7 percent of the actual rate (based on the 
findings for program group members), then a total of 18.6 percent of the control group members 
attended an orientation (i.e., 11.5 divided by .617). Since this includes the 7.5 percent of control 
group members who also participated in employment-related activities and, thus, were assumed to have 
attended orientation, the remaining 11.1 percent of those who attended orientation did not participate 
in any other program activities. 

Finally, both the 24-month survey data and the casefile data indicated that approximately 1 
percent of the control group members were referred for a sanction (a reduction in their AFDC grant) 
because they dia not participate in a prescribed Project Independence activity and did not provide an 
acceptable excuse. Given the similarity between the survey and casefile estimates, no correction was 
made, and the casefile estimate of 1.2 percent was used as the rate at which control group members 
were referred for a sanction. None of these sanctioned control group members attended orientation 
or participated in employment- related activities through Project Independence. 



rV. Summary 

Corrections based on a review of Project Independence casefiles indicate that 19.8 percent of 
control group members were exposed to some aspect of the Project Independence treatment. This 
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estimate includes three elements. First, an estimated 11.1 percent of control group members attended 
Project Independence orientation but did not participate in any Project Independence employment- 
related activities. An additional 1.2 percent of the control group were referred for a sanction for not 
participating in a prescribed activity and were not exposed to any other aspect of the program. 
Finally, the most intensive involvement in Project Independence was probably limited to 7.5 percent 
of control group members. These were individuals who participated in at least one employment-related 
activity through Project Independence. The sum of these three components — 11.1 plus 1.2 plus 7.5 
- yields the estimate of 19.8 percent of control group members who were exposed to some aspect of 
the Project Independence treatment. This is the most reliable estimate of the control group's exposure 
to Project Independence and is the one that was adopted for this report. 
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APPENDIX D 
SUPPLEMENTAL TABLE TO CHAPTER 4 




TABLE D.I 



AVERAGE NUMBER OF MONTHS OF PARTICIPATION IN PROJECT INDEPENDENCE 
AND NON-PROJECT INDEPENDENCE EMPLOYMENT -RELATED ACTIVITIES 
WITHIN FIVE YEARS AFTER RANDOM ASSIGNMENT 



Activity Source and Participation Measure 



Program 
Group 



Control 
Group 



Project Independence and 

non-Project Independence sources combined 

Ever participated in any employment- 
related activity (a) 



7.05 



5.51 



Ever participated in independent 
job search or job club (a) 

Independent job search 

Job club 



1.66 
1.41 
0.26 



0.94 
0.84 
0.13 



Ever participated in education or training (a) 
ABE or GEO 
ESL 

Vocational training or post -secondary education 
On-the-job training 



6.38 
1.10 
0.30 
4.32 
0.38 



4.99 
0.82 
0.51 
3.16 
0.34 



Project Independence activities 



Ever participated in any employment - 
related activity (a) 



4.73 



1.03 



Ever participated in independent 
job search or job club (a) 

independent job search 

Job club 



1.41 
1.20 
0.23 



0.30 
0.27 
0.04 



Ever participated in education or training (a) 
ABE or GEO 
ESL 

Vocational training or post-secondary education 
On-the-job training 



2.31 
0.38 
0.16 
1.35 
0.22 



0.36 
0.07 
0.02 
0.21 
0.03 



Sample size (total = 1 .029) 



520 



509 



SOURCES: The 24- 
casefiles. 



•month survey of a subsample of program and control group members and Project Independence 



NOIES: Length of participation is defined as the number of months with any participation in a given activity. 

(a) Individuals could participate in more than one activity during the follow-up period: therefore, 
the sum of percentages in specific activities may exceed the category percentage. 
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APPENDK E 
SUPPLEMENTAL TABLE TO CHAPTER 5 



TABLE E.1 



TWO-YEAR IMPACTS OF PROJECT INDEPENDENCE ON 
EMPLOYMENT, EARNINGS, AFDC RECEIPT, AFDC PAYMENTS, 
FOOD STAMP RECEIPT, AND FOOD STAMP PAYMENTS FOR THE FULL SAMPLE 



Outcome and Foitow-up Period 



Program Group Control Group Difference 



Percentage 
Difference 



Ever employed (%) 

Quarters 2-9 

And employed in quarter 9 
And not employed in quarter 9 

Quarters 2-5 

And employed In quarter 5 
And not employed in quarter 5 

Quarters 6-9 

And employed In quarter 9 
And not employed in quarter 9 

Average quarterly employment rate (%) 
Quarters 2-9 
Quarters 2-5 
Quarters 6-9 



66.3 
38.3 
28.1 
55.4 
36.7 
18.7 
53.3 
38.3 
15.1 



37.3 
36.8 
37.8 



63.8 
37.8 
26.0 
52.6 
34.6 
18.0 
51.2 
37.3 
13.4 



35.5 
34.5 
36.5 



2.5 
0.4 
2.1 
2.8 
2.2 
0.7 
2.1 
0.4 
1.7 



1 .8 *** 
2.3 *** 
1.3 ** 



4.0% 
1.1% 
8.1% 
5.4% 
6.2% 
3.7% 
4.1% 
1.1% 
12.7% 



5.1% 
6.6% 
3.7% 



Employed in all quarters (%) 
Quarters 2-5 
Quarters 6-9 



19.2 
23.1 



17.4 
22.1 



1.9 
1.0 



10.7% 
4.5% 



First employed (%) 
Quarters 2-5 
Quarters 6-9 

Ever employed (%) 
Prior quarter 4 
Prior quarter 3 
Prior quarter 2 
Prior quarter 1 

Quarter of random assignment 

Quarter 2 

Quarter 3 

Quarter 4 

Quarter 5 

Quarter 6 

Quarter 7 

Quarter 8 

Quarter 9 

Quarter 1 0 

Quarter 1 1 

Quarter 1 2 

Quarter 13 

Average total earnings ($) 
Quarters 2-9 
Quarters 2-5 
Quarters 6-9 



55.4 
10.9 



43.3 
44.2 
44.1 
42.0 
40.1 
36.5 
37.2 
36.7 
36.7 
37.4 
37.6 
38.1 
38.3 
n/a 
n/a 
n/a 
n/a 



5766 
2548 
3219 



52.6 
11.2 



43.7 
44.6 
43.2 
42.0 
39.9 
33.8 
34.8 
34.9 
34.6 
34.9 
36.6 
36.7 
37.8 
n/a 
n/a 
n/a 
n/a 



5539 
2401 
3138 



2.8 *** 

-0.3 



-0.4 
-0.5 
0.9 
0.0 
0.3 
2.7 
2.4 
1.8 
2.2 
2.6 
1.0 
1.4 
0.4 
n/a 
n/a 
n/a 
n/a 



*** 

*** 

** 

*** 

*** 



5.4% 
-2.5% 



-0.9% 
-1.1% 
2.0% 
0.0% 
0.7% 

e.1% 

6.9% 
5.0% 
6.2% 
7.4% 
2.8% 
3.7% 



227 * 
146 ** 
80 



.1% 
n/a 
n/a 
n/a 
n/a 



4.1% 
6.1% 
2.6% 



Average earnings per quarter employed ($) 
Quarters 2-9 
Quarters 2-5 
Quarters 6-9 



1931 
1731 
2126 



1950 
1739 
2149 



-18 (a) 
-6 (a) 
-23 (a) 



-0.9% 
-0.4% 
-1.1% 



(continued) 
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TABLE E.l (continued) 



Outcome and Folbw-up Period 



Program Group Control Group 



Difference 



Percentage 
Difference 



Average total earnings ($) 



Prior quarter 4 


837 


842 


-5 


-0.6% 


Prior quarter 3 


845 


859 


-14 


-1.6% 


Prior quarter 2 


801 


799 


2 


0.2% 


Prior quarter 1 


695 


711 


-16 


-2.2% 


Quarter of random assignment 


409 


422 


-13 


-3.1% 


Quarter 2 


507 


485 


22 


4.6% 


Quarters 


643 


580 


63 *** 


10.8% 


Quarter 4 


681 


650 


30 


4.7% 


Quarter 5 


717 


686 


31 


4.6% 


Quarters 


754 


720 


33 


4.6% 


Quarter 7 


/ 9<i 


fOf 


35 


4.6% 


Quarters 


824 


812 


13 


1 .5% 


Quarter 9 


849 


849 


0 


0.0% 


Quarter 1 0 


n/a 


n/a 


n/a 


n/a 


Quarter 1 1 


n/a 


n/a 


n/a 


n/a 


Quarter 12 


n/a 


n/a 


n/a 


n/a 


Quarter 13 


n/a 


n/a 


n/a 


n/a 


er received any AFDC payments (%) 










Quarters 2-9 


88.3 


89.5 


-1.1 ** 


-1.3% 


Quarters 2-5 


85.2 


86.7 


-1.5 *** 


- 1 .7% 


Quarters 6-9 


68.6 


71.3 


—2 7 *** 


-3.8% 


er off AFDC (%) 










Quarters 2-9 


62.8 


58.7 


4.1 *** 


6.9% 


And did not receive AFDC in quarter 9 


48.8 


46.4 


2 4 *** 


5.2% 


And received AFDC in quarter 9 


14.0 


12.4 


1.6 *** 


13.2% 


Quarters 2-5 


43.4 


37.9 


5.5 *** 


14.5% 


Quarters 6-9 


55.7 


52.9 


2 Y *** 


5.2% 



First off AFDC (%) 
Quarters 2-5 
Quarters 6-9 

Average number of months 
receiving AFDC payments 

Quarters 2-9 

Quarters 2-5 

Quarters 6-9 



43.4 
19.4 



14.20 
7.82 
6.39 



37.9 
20.9 



15.06 
8.31 
6.75 



5.5 *** 
-1.4 ** 



-0.85 *** 
-0.49 *** 
-0.36 *** 



14.5% 
-6.8% 



-5.7% 
-5.9% 
-5.4% 



Ever received any AFDC payments (%) 
Prior quarter 4 
Prior quarter 3 
Prior quarter 2 
Prior quarter 1 

Quarter of random assignment 

Quarter 2 

Quarters 

Quarter 4 

Quarters 

Quarter 6 

Quarter 7 

Quarter 8 

Quarter 9 

Quarter 10 

Quarter 1 1 

Quarter 12 

Quarter 13 



22.9 
22.9 
22.4 
20.4 
53.6 
79.7 
72.4 
67.0 
64.6 
62.3 
59.5 
55.7 
51.2 
47.5 
n/a 
n/a 
n/a 



22.8 
22.9 
22.1 
20.2 
54.1 
81.7 
76.3 
71.7 
68.7 
65.4 
62.5 
58.6 
53.6 
49.2 
n/a 
n/a 
n/a 



0.1 

0.1 

0.4 

0.2 
-0.6 
-2.0 
-3.9 
-4.7 
-4.2 
-3.1 
-3.0 *** 
-2.9 
-2.4 

-i;/ 

n/a 
n/a 
n/a 



*** 
*** 
*** 

*** 



*** 
*** 



0.4% 
0.2% 
1 .6% 

-1.0% 
-2.4% 
-5.1% 
-6.5% 
-6.0% 
-4.7% 
-4.8% 
-4.9% 
-4.5% 
-3.6% 
n/a 
n/a 
n/a 



(continued) 
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TABLE E.I (continued) 



Outcome and Folbw-up Period 



Program Group Control Group Difference 



Percentage 
Difference 



Average total AFDC payments received ($) 
Quarters 2-9 
Quarters 2 -5 
Quarters 6-9 

Average AFDC payment per month received ($) 
Quarters 2-9 
Quarters 2-5 
Quarters 6-9 

Average AFDC payments received {$) 
Prior quarter 4 
Prior quarter 3 
Prior quarter 2 
Prior quarter 1 

Quarter of random assignment 

Quarter 2 

Quarter 3 

Quarter 4 

Quarter 5 

Quarter 6 

Quarter 7 

Quarters 

Quarter 9 

Quarter 10 

Quarter 1 1 

Quarter 1 2 

Quarter 1 3 

Ever received any Food Stamps (%) 
Quarters 2-9 
Quarters 2-5 
Quarters 6-9 

Ever off Food Stamps (%) 

Quarters 2-9 

And did not receive Food Stamps in quarter 9 
And received Food Stamps in quarter 9 

Quarters 2-5 

Quarters 6-9 

First off Food Stamps (%) 
Quarters 2-5 
Quarters 6-9 



4028 
2196 
1832 



264 
261 
267 



174 
173 
169 
151 
321 
626 
542 
518 
510 
501 
480 
444 
407 
376 
n/a 
n/a 
n/a 



90.5 
88.6 
72.6 



55.6 
43.2 
12.4 
36.6 
49.7 



36.6 
18.9 



4293 
2348 
1945 



265 
263 
266 



176 
175 
173 
153 
327 
654 
584 
565 
546 
531 
509 
473 
432 
393 
n/a 
n/a 
n/a 



91.0 
89.1 
73.9 



54.0 
42.6 
11.5 
34.6 
48.4 



34.6 
19.4 



-265 *** 
-152 *** 
-113 *** 



-2 (a) 
-2 (a) 
-1 (a) 



-2 
-2 
-4 
-2 
-6 

—27 *** 
-43 *** 
-46 *** 
-36 *** 
-30 *** 
-29 *** 
-29 *** 
-25 *** 
-18 ** 

n/a 

n/a 

n/a 



-0.5 
-0.5 
-1.3 



1 .5 
0.6 
0.9 
2.0 
1.3 



2.0 
-0.5 



-6.2% 
-6.5% 
-5.8% 



-0.5% 
-0.6% 
-0.4% 



-1.0% 
-1.1% 
-2.2% 
-1.5% 
-1.9% 
-4.2% 
-7.3% 
-8.2% 
-6.6% 
-5.7% 
-5.8% 
-6.1% 
-5.7% 
-4.5% 
n/a 
n/a 
n/a 



-0.6% 
-0.6% 
-1.8% 



2.8% 
1 .4% 
8.0% 
5.8% 
2.7% 



5.8% 
-2.6% 



Average number of months 
receiving Food Stamps 

Quarters 2-9 

Quarters 2-5 

Quarters 6-9 



15.36 
8.48 
6.88 



15.70 
8.66 
7.04 



-0.34 *** 
-0.19 *** 
-0.16 * 



-2.2% 
-2.2% 
-2.2% 



(continued) 
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TABLE E.I (continued) 



vjuicome ana roiiow— up renoa 


rrogram oroup 


Control Group 


Difference 


Percentage 
Difference 


Ever received any Food Stamps (%) 










Prior quarter 4 


33.5 


33.7 


—0.2 


—0.5% 


Prior cjuarter 3 


35 .4 


O IT O 

35 .8 


—0.4 


— 1 .1% 


Prior quarter 2 


37.3 


37.8 


—0.4 


— 1 .1% 


Prior quarter 1 




42.5 


0.0 


0.0% 


Quarter of random assignment 


84.2 


84.3 


—0.1 


—0.1% 


uuarler 2 


84.1 


84.5 


—0.4 


—0.4% 


Quarter 3 


77.0 


78.5 


— 1 .5 ** 


— 1 .9% 


Quarter 4 


73.2 


74.7 


— 1 .4 ** 


-1 .9% 


Quarter 5 


70.5 


71 .8 


— 1 .3 


— 1 .8% 


Quarter 6 


67.4 


69.0 


— 1 .6 ** 


—2.3% 


Quarter 7 


64.0 


65.5 


-1.6 ** 


-2.4% 


Quarter 8 


59.9 


61.4 


-1.5 * 


-2.5% 


Quarter 9 


OD.O 


0/.4 


—0.6 


— 1 .0% 


Quarter 1 0 


54.1 


54.3 


—0.1 


—0.3% 


Quarter 1 1 


n/a 


n/a 


n/a 


n/a 


Quarter 1 2 


n/a 


n/a 


n/a 


n/a 


Quarter 13 


n/a 


n/a 


n/a 


n/a 


Average total value of Food Stamps received ($) 










Quarters 2—9 


4002 


41 38 


— 136 ** 


—3.3% 


Quarters 2-5 


2148 


2213 


-66 *** 


-3.0% 


Quarters 6-9 


1855 


1925 


-70 *** 


-3.6% 


Aversge VQfue of Food StQcnps por month received ($) 










Quarters 2-^9 


2oi 




O /a) 


i i o/ 

— 1 . 1 76 


Quarters 2^5 






-2 fa; 


— O.ovo 


Quarters 6^9 






-4 fa; 


— 1 .OA> 


Average value of Food Stamps received ($) 










Prior quarter 4 


201 


203 


—1 


—0./^% 


Prior quarter 3 


214 


<i19 


— O 


o oo/ 


Prior quarter 2 


228 


231 


—3 


— 1 .4% 


Prior quarter 1 


249 


251 


—2 


rv CO/ 


Quarter of random assignment 


470 


470 


— 1 


—0.1 % 


Quarter 2 


581 


590 


r\ it 
— 9 * 


— 1 .5% 


Quarter 3 


536 


554 


— 17 


—3.1% 


Quarter 4 


521 


541 




—3.8% 


Quarter 5 


510 


529 


— 1 y 


O fiO/ 


Quarter 6 


502 


O 1 o 


— 1 D ''^ 


O 00/ 


Quarter 7 


476 


498 


—21 *** 


-4.3% 


Quarter 8 


453 


471 


-18 *** 


-3.9% 


Quarter 9 


424 


438 


-14 ** 


-3.2% 


Quarter 10 


387 


394 


-6 


-1.6% 


Quarter 1 1 


n/a 


n/a 


n/a 


n/a 


Quarter 12 


n/a 


n/a 


n/a 


n/a 


Quarter 1 3 


n/a 


n/a 


n/a 


n/a 


Sample size (total = 18,233) 


13.509 


4.724 







SOURCES: 



See Table 5.1. 



NOTES: See Table 5.1. 

Italics indicate a nonexperimental estimate. 
"N/a" indicates that data are not available. 

(a) This comparison is nonexperimental. No test of statistical significance was performed. 
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Friedlander, Gary Burtless. 

Canada's Self-Sufficiency Project (SSP) 

A test of the effectiveness of a temporary earnings supplement on the employment and welfare receipt of 
public assistance recipients. 

Making Work Pay Better Than Welfare: An Early Look at the Self-Sufficiency Project (Social Research and 
Demonstration Corporation). 1994. Susanna Lui-Gurr, Sheila Currie Vernon^ Tod Mijanovich. 

The Saturation Work Initiative Model (SWIM) 

A test of the feasibility and effectiveness of an ongoing participation requirement in a welfare-to-work 
program. 

Interim Report on the Saturation Work Initiative Model in San Diego. 1988. Gayle Hamilton. 
Firuil Report on the Saturation Work Initiative Model in San Diego, 1989. Gayle Hamilton, Daniel Friedlander. 
The Saturation Work Initiative Model in San Diego: A Five-Year Follow-up Study, 1993. Daniel Friedlander, Gayle 
Hamilton. 

The Demonstration of State Work/Welfare Initiatives 

A test of the feasibility and effectiveness of various state employment initiatives for welfare recipients. 

Arizona: Preliminary Management Lessons from the WIN Demonstration Program. 1984. Kay Sherwood. 

Arkansas: Interim Findings from the Arkansas WIN Demonstration Program. 1984. Janet (Juint. 
Final Report on the WORK Program in Two Counties. 1985. Daniel Friedlander, Gregory Hoerz, Janet Quint, 
James Riccio. 

California: Preliminary Findings from the San Diegc Job Search and Work Experience Demonstration. 1984. 

Barbara Goldman, Judith Gueron, Joseph Ball, Marilyn Price. 
Findings from the San Diego Job Search and Work Experience Demonstration. 1985. Barbara Goldman, Daniel 

Friedlander, Judith Gueron, David Long. 
Final Report on the San Diego Job Search and Work Experience Demonstration. 1986. Barbara Goldman, Daniel 

Friedlander, David Long. 

Illinois: Interim Findings from the WIN Demonstration Program in Cook County. 1986. Janet Quint, Cynthia (juy. 
Final Report on Job Search and Work Experience in Cook County. 1987. Daniel Friedlander, Stephen Freedman, 
Gayle Hamilton, Janet Quint. 

Maine: Interim Findings from a Grant Diversion Program. 1985. Patricia Auspos. 

Final Report on the Training Opportunities in the Private Sector Program. 1988. Patricia Auspos, George Cave, 
David Long. 

Maryland: Interim Findings from the Maryland Employment Initiatives Programs. 1984. Janet Quint. 
Final Report on the Employment Initiatives Evaluation. 1985. Daniel Friedlander, Gregory Hoerz, David Long, 
Janet Quint. 

Supplemental Report on the Baltimore Options Program. 1987. Daniel Friedlander. 

New Jersey: Final Report on the Grant Diversion Project. 1988. Stephen Freedman, Jan Bryant, George Cave. 

Virginia: Interim Findings from the Virginia Employment Services Program. 1985. Marilyn Price. 
Final Report on the Virginia Employment Services Program. 1986. James Riccio, George Cave, Stephen Freedman, 
Marilyn Price. 

West Virginia: Interim Findings on the Community Work Experience Demonstrations. 1984. Joseph Ball. 
Final Report on the Community Work Experience Demonstrations. 1986. Daniel Friedlander, Marjorie Erickson, 
Gayle Hamilton, Virginia Knox. 

Other Reports on the Demonstration of State Work/Welfare Initiatives 

Documentation of the Data Sources and Analytical Methods Used in the Benefit-Cost Analysis of the EPP/EWEP 

Program in San Diego. 1985. David Long, Virginia Knox. 
Relationship Between Earnings and Welfare Benefits for Working Recipients: Four Area Case Studies. 1985. Barbara 

Goldman, Edward Cavin, Marjorie Erickson, Gayle Hamilton, Darlene Hasselbring, Sandra Reynolds. 
Welfare Grant Diversion: Early Observations from Programs in Six States. 1985. Michael Bangser, James Healy, 

Robert Ivry. 
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A Survey of Panicipams and Worksite Supervisors in the New York City Work Experience Program. 1986. Gregory 
Koerz, Karla Hansen. 

Welfare Grant Diversion: Lessons and Prospects. 1986. Michael Bangser, Janies Healy, Robert Ivry. 
Work Initiatives for Welfare Recipients: Lessons from a Multi-State Experiment. 1986. Judith Gueron. 

The Subgroup/Performance Indicator Study 

A study of the impacts of selected welfare-to-work programs on subgroups of the AFDC caseload. 

A Study of Performance Measures and Subgroup Impacts in Three Welfare Employment Programs. 1987. Daniel 
Friedlander, David Long. 

Subgroup Impacts and Performance Indicators for Selected Welfare Employment Programs. 1988. Daniel Friedlander. 

The Self-Employment favestment Demonstration (SEID) 

A test of the feasibility of operating a program to encourage self-employment among recipients of AFDC. 

Self'Employment for Welfare Recipients: Implementation of the SEID Program. 1991. Cynthia Guy, Fred Doolittle, 
Barbara Fink. 

The WIN Research Laboratory Project 

A test of innovative service delivery approaches in four Work Incentive Program (WIN) offices. 

Immediate Job Search Assistance: Preliminary Results from the Louisville WIN Research Laboratory Project. 1980. 
Barbara Goldman. 

Preliminary Research Findings: WIN Research Laboratory Project. 1980. MDRC. 

Final Report on WIN Services to Volunteers: Denver WIN Research Laboratory Project. 1981 . Ellen Slaughter, 

Paulette Turshak, Gaie Whiteneck, Edward Baumheier. 
Impacts of the Immediate Job Search Assistance Experiment: Louisville WIN Research Laboratory Project. 1981. 

Barbara Goldman. 

The Workings of WIN: A Field Observation Study of Three Local Offices. 1981. Sydelle Levy. 
Welfare Women in a Group Job Search Program: Their Experiences in the Louisville WIN Research Laboratory- 
Project. 1982. Joanna Gould-Stuan. 
The WIN Labs: A Federal/Local Partnership in Social Research. 1982. Joan Leiman. 

Job Search Strategies: Lessons from the Louisville WIN Laboratory. 1983. Carl Wolfhagen, Barbara Goldman. 



THE PARENTS' FAIR SHARE DEMONSTRATION 

A demonstration aimed at reducing child poverty by increasing the job-holding, earnings, and child support 
payments of unemployed, noncustodial parents (usually fathers) of children receiving public assistance. 

Caring and Paying: What Fathers and Mothers Say About Child Support. 1992. Frank Furstenberg, Jr., Kay 

Sherwood, Mercer Sullivan. 
Qiild Support Enforcement: A Case Study. Working Paper. 1993. Dan Bloom. 

Matching Opportunities to Obligations: Lessons for Child Support Reform from the Parents' Fair Share Pilot Phase. 
1994. Dan Bloom, Kay Sherwood. 



PROGRAMS FOR TEENAGE PARENTS ON \\TELFARE 
The LEAP Evaluation 

An evaluation of Ohio's Learning, Earning, and Parenting (LEAP) Program, which uses financial incentives to 
encourage teenage parents on welfare to stay in or return to school. 

LEAP: Implementing a Welfare Initiative to Improve School Attendance Among Teenage Parents. 1991. Dan Bloom, 

Hilary Kopp, David Long, Denise Polii. 
LEAP: Interim Findings on a Welfare Initiative to Improve School Aaendance Among Teenage Parents. 1993. Dan 

Bloom, Veronica Fellerath, David Long, Robert Wood. 
LEAP: The Educational Effects of LEAP and Enhanced Sen ices in Cleveland. 1994. David Long, Robert Wood, 

Hilary Kopp. 

The New Chance Demonstration 

A lest of a comprehensive program of services that seeks to improve the economic status and general wcll-beine of 
a group of highly disadvantaged young women and their children. 
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New Chance: Implementing a Comprehensive Program for Disadvantaged Young Mothers and Their Children. 1991. 

Janet Quint, Barbara Fink, Sharon Rowser. 
New Chance: An Innovative Program for Young Mothers and Their Children. Brochure. 1993. 
Lives of Promise, Lives of Pain: Young Mothers After New Chance. Monograph. 1994. Janet Quint, Judith Musick, 

with Joyce Ladner. 

New Chance: Interim Findings on a Comprehensive Program for Disadvantaged Young Mothers and Their Children. 

1994. Janet Quint, Denise Polit, Hans Bos, George Cave. 
New Chance: The Cost Analysis of a Comprehensive Program for Disadvantaged Young Mothers and Their Children. 

1994. Barbara Fink. 

Project Redirection 

A test of a comprehensive program of services for pregnant and parenting teenagers. 

The Challenge of Serving Teenage Mothers: Lessons from Project Redirection. Monograph. 1988. Denise Polit, Janet 
Quint, James Riccio. 

The Community Service Projects 

A test of a New York State teenage pregnancy prevention and services initiative. 

The Community Service Projects: A New York State Adolescent Pregnancy Initiative. 1986. Cynthia Guy. 
The Community Service Projects: Final Report on a New York State Adolescent Pregnancy Prevention and Services 
Program. 1988. Cynthia Guy, Lawrence Bailis, David Palasits, Kay Sherwood. 

THE SCHOOL-TO-WORK TRANSITION PROJECT 

A study of innovative programs that help students make the transition from school to work. 

The School'tO'Work Transition and Youth Apprenticeship: Lessons from the U.S. Experience. 1993. Thomas Bailey, 
Donna Merritt. 

Homegrown Lessons: Innovative Programs Linking School and Work (Jossey-Bass Publishers). Book. 1995. Edward 

Pauly, Hilary Kopp, Joshua Haimson. Revised version of a 1994 MDRC report. 
Learning Through Work: Designing and Implementing Quality Worksite Learning for High School Students. 1994. 

Susan Goldberger, Richard Kazis, Mary Kathleen OTlanagan (all of Jobs for the Fuuire). 



THE NATIONAL JTPA STLT)Y 

A study of 16 local programs under the Job Training Partnership Act (JTPA), the nation *s job training system for 
low-income individuals. 

Implementing the National JTPA Study. 1990. Fred Doolittle, Linda Traeger. 

The National JTPA Study: Site Characteristics and Participation Patterns. 1993. James Kemple, Fred Doolittle, John 
Wallace. 

A Summary of the Design and Implementation of the National JTPA Study. 1993. Fred Doolittle. 

THE NATIONAL SUPPORTED WORK DEMONSTRATION 

A test of a transitional work experience program for four disadvantaged groups. 

Summary and Findings of the National Supported Work Demonstration. 1980. MDRC Board of Directors. 



About MDRC 



The Manpower Demonstration Research Corporation (MDRC) is a 
nonprofit social policy research organization founded in 1974 and 
located in New York City and San Francisco. Its mission is to design 
and rigorously field-test promising education and employment-related 
programs aimed a: improving the well-being of disadvantaged adults 
and youth, and to provide policymakers and practitioners with reliable 
evidence on the effectiveness of social programs. Through this work, 
and its technical assistance to program administrators, MDRC seeks to 
enhance the quality of public policies and programs. MDRC actively 
disseminates the results of its research through its publications and 
through interchange with policymakers, administrators, practitioners, 
and the public. 

Over the past two decades - working in partnership with more than 
forty states, the federal government, scores of commimities, and 
numerous private philanthropies - MDRC has developed and studied 
more than three dozen promising social policy initiatives. 
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